EE 5359 – MULTIMEDIA PROCESSING Subrahmanya Maira Venkatrav 1000615952Project Proposal: Sub pixel motion estimation for side information generation in Wyner-Ziv decoder.Wyner-Ziv(WZ) encoder is a low -complexity encoder and can be made to achieve compression comparable to traditional high complexity encoders but at the expense of a high-complexity decoder. The high complexity of the decoder is mainly attributed to the generation of side information which involves motion estimation. The quality of the frame reconstructed at the decoder mainly depends on quality of the motion estimation. Hence in this proposal sub pixel motion estimation is considered for side information generation.1.Overview of WZ EncoderThe WZ encoding process involves encoding of key frames and WZ frames. The overall encoding process is illustrated in Figure 1. Some of the input frames are marked as key frames and encoded using H.264 Intra (I) frame[6][18]encoding. The WZ frames are encoded as follows: The difference between previous reconstructed key frame and WZ frame is quantized using a uniform scalar quantizer and the output is encoded using low-density-parity-check accumulated (LDPCA) code. Figure 1. Block diagram of WZ encoder2.Overview of WZ DecoderThe first step in decoding a WZ frame is generation of side information (SI) using key frames. The SI frame generated is used by the low density parity check accumulate (LDPCA) decoder to decode the WZ bitstream and generate the WZ frame. The previous key frame is subtracted from the SI frame generated to produce error frame which is subsequently quantized. This is used by LDPCA decoder to correct bit errors in WZ encoded error frame. The error frame obtained in this way is added to the key frame and de-quantized to reconstruct the WZ frame. At low bitrates some of the macroblocks in the WZ frame cannot be reconstructed. These macroblocks are replaced with the corresponding macroblocks from WZ estimated frame.Figure 2. Block diagram of WZ decoder[1]3.Generation of Side Information (SI)The SI frame generation is key aspect of WZ decoding process. The quality of the decoding is dependent on the SI frame and in terms of complexity this is a major component of WZ decoder.Figure 3.Side information generation using key frames[1]The generation of SI is illustrated in Figure 3 and it involves,- Motion estimation (ME) between two key frames to obtain motion vectors (MV). The estimation is done in both forward and backward directions to obtain MVFand MVBrespectively as shown in Figure 3. The block sizes are used for ME are 16x16, 8x8 and 4x4. - The derivation of motion vectors for WZ frames. This is done by scaling MVs obtained in the previous step by the ratio calculated as distance between WZ frame to previous key frame to the distance between key frames themselves. In Figure 3, the scaling factor is ½ since the ratio of distance between key frame and WZ frame to distance between two key frames is ½. - Obtaining the estimation for macroblock of a WZ frame by interpolation of macroblocks from the previous and next key frames. The motion vectors calculated in the previous step are used here to obtain mapping of macroblocks in WZ frame to key frame macroblocks. The forward predicted frame (PF) is obtained using forward motion vector MVFand backward predicted frame (PB) is obtained using backward motion vector MVB.Then the side information frame Y is obtained as (PF+PB)/2 as shown in Figure 3.4.Sub-pixel motion estimation for SI generationThe side information generated can be improved by using sub-pixel motion vectors for both forward and backward predictions. In order to derive these sub-pixel positions interpolation between pixels needs to be performed. For half pixel motion estimation there are three pixel positions that need to be evaluated. For quarter pixel motion estimation there are twelvepixel positions that need to be evaluated.The generation of sub-pixel positions is done as per H.264 standard[8]and is briefly describedbelow: 1. Half-pixel positions:In Figure 4 the pixel positions numbered H33, G33 and D33 are half pixel positions and need to be derived.Figure 4. Full and half pixel positionsThese are generated by interpolating full pixel or half pixel values using a six tap filter [1 -5 20 20 -5 1]/32. Following equations can be used, H33 = [F13 + -5 * F23 + 20 * F33 + 20 * F43 + -5 * F53 + F63 + 15] >> 5 G33 = [F31 + -5 * F32 + 20 * F33 + 20 * F34 + -5 * F35 + F36 + 15] >> 5 D33 = [H31 + -5 * H32 + 20 * H33 + 20 * H34 + -5 * H35 + H36 + 15] >> 52. Quarter-pixel positions:The quarter pixels are obtained by averaging nearest full pixel or half pixel positions.Figure 5. Full, half and quarter pixel positions The following equations are used for obtaining quarter pixel positions, q1 = ( F33 + G33 + 1 ) >> 1 q2 = ( G33 + F34 + 1 ) >> 1 q3 = ( F33 + H33 + 1 ) >> 1 q4 = ( H33 + G33 + 1 ) >> 1 q5 = (G33 + D33 + 1 ) >> 1 q6 = ( G33 + H34 + 1 ) >> 1 q7 = ( H33 + D33 + 1 ) >> 1 q8 = ( D33 + H34 + 1 ) >> 1 q9 = ( H33 + F43 + 1 ) >> 1 q10 = ( H33 + G43 + 1 ) >> 1 q11 = ( D33 + G43 + 1 ) >> 1 q12 = ( G43 + H34 + 1 ) >> 1 The forward and backward predicted data obtained for each partition block is averaged to obtain the final prediction block. In case for a block if there is no motion vector, then intra prediction can be used to predict the block from neighboring pixels. The improvement in the quality of SI generated with the sub pixel motion estimation over full pixel motion estimation can be measured both visually and quantitatively[17]. The quantitative measurement can be done by PSNR of the predicted frame with reference to the original frame. The objective is to get a good improvement in the quality of SI frame.3. Results:The half-pel motion estimation is implemented for WZ frame generation using JM reference software. The even frames are encoded as I frames and odd frames are encoded using WZ encoder. For quality comparison between WZ encoder and H.264 encoder a separate encoding is done with even frame being I frame and odd frame being P frame. The WZ frame obtained using SI prediction is analyzed for PSNR with reference to correspondingH.264 P frame. The average PSNR plot for a QCIF (176x144) test sequence for a ME search range of 64 is shown in Table 1 and the plot is shown in Figure 6.Coastguard_qcif.yuv (Search Range-64)SI prediction scheme
View Full Document
Unlocking...