EE 5359 MULTIMEDIA PROCESSING Subrahmanya Maira Venkatrav 1000615952 Project Proposal Sub pixel motion estimation for side information generation in WynerZiv decoder Wyner Ziv WZ encoder is a low complexity encoder and can be made to achieve compression comparable to traditional high complexity encoders but at the expense of a high complexity decoder The high complexity of the decoder is mainly attributed to the generation of side information which involves motion estimation The quality of the frame reconstructed at the decoder mainly depends on quality of the motion estimation Hence in this proposal sub pixel motion estimation is considered for side information generation 1 Overview of WZ Encoder The WZ encoding process involves encoding of key frames and WZ frames The overall encoding process is illustrated in Figure 1 Some of the input frames are marked as key frames and encoded using H 264 Intra I frame 6 18 encoding The WZ frames are encoded as follows The difference between previous reconstructed key frame and WZ frame is quantized using a uniform scalar quantizer and the output is encoded using low density parity check accumulated LDPCA code Figure 1 Block diagram of WZ encoder 2 Overview of WZ Decoder The first step in decoding a WZ frame is generation of side information SI using key frames The SI frame generated is used by the low density parity check accumulate LDPCA decoder to decode the WZ bitstream and generate the WZ frame The previous key frame is subtracted from the SI frame generated to produce error frame which is subsequently quantized This is used by LDPCA decoder to correct bit errors in WZ encoded error frame The error frame obtained in this way is added to the key frame and de quantized to reconstruct the WZ frame At low bitrates some of the macroblocks in the WZ frame cannot be reconstructed These macroblocks are replaced with the corresponding macroblocks from WZ estimated frame Figure 2 Block diagram of WZ decoder 1 3 Generation of Side Information SI The SI frame generation is key aspect of WZ decoding process The quality of the decoding is dependent on the SI frame and in terms of complexity this is a major component of WZ decoder Figure 3 Side information generation using key frames 1 The generation of SI is illustrated in Figure 3 and it involves Motion estimation ME between two key frames to obtain motion vectors MV The estimation is done in both forward and backward directions to obtain MVF and MVB respectively as shown in Figure 3 The block sizes are used for ME are 16x16 8x8 and 4x4 The derivation of motion vectors for WZ frames This is done by scaling MVs obtained in the previous step by the ratio calculated as distance between WZ frame to previous key frame to the distance between key frames themselves In Figure 3 the scaling factor is since the ratio of distance between key frame and WZ frame to distance between two key frames is Obtaining the estimation for macroblock of a WZ frame by interpolation of macroblocks from the previous and next key frames The motion vectors calculated in the previous step are used here to obtain mapping of macroblocks in WZ frame to key frame macroblocks The forward predicted frame PF is obtained using forward motion vector MVF and backward predicted frame PB is obtained using backward motion vector MVB Then the side information frame Y is obtained as PF PB 2 as shown in Figure 3 4 Sub pixel motion estimation for SI generation The side information generated can be improved by using sub pixel motion vectors for both forward and backward predictions In order to derive these sub pixel positions interpolation between pixels needs to be performed For half pixel motion estimation there are three pixel positions that need to be evaluated For quarter pixel motion estimation there are twelve pixel positions that need to be evaluated The generation of sub pixel positions is done as per H 264 standard 8 and is briefly described below 1 Half pixel positions In Figure 4 the pixel positions numbered H33 G33 and D33 are half pixel positions and need to be derived Figure 4 Full and half pixel positions These are generated by interpolating full pixel or half pixel values using a six tap filter 1 5 20 20 5 1 32 Following equations can be used H33 F13 5 F23 20 F33 20 F43 5 F53 F63 15 5 G33 F31 5 F32 20 F33 20 F34 5 F35 F36 15 5 D33 H31 5 H32 20 H33 20 H34 5 H35 H36 15 5 2 Quarter pixel positions The quarter pixels are obtained by averaging nearest full pixel or half pixel positions Figure 5 Full half and quarter pixel positions The following equations are used for obtaining quarter pixel positions q1 F33 G33 1 1 q2 G33 F34 1 1 q3 F33 H33 1 1 q4 H33 G33 1 1 q5 G33 D33 1 1 q6 G33 H34 1 1 q7 H33 D33 1 1 q8 D33 H34 1 1 q9 H33 F43 1 1 q10 H33 G43 1 1 q11 D33 G43 1 1 q12 G43 H34 1 1 The forward and backward predicted data obtained for each partition block is averaged to obtain the final prediction block In case for a block if there is no motion vector then intra prediction can be used to predict the block from neighboring pixels The improvement in the quality of SI generated with the sub pixel motion estimation over full pixel motion estimation can be measured both visually and quantitatively 17 The quantitative measurement can be done by PSNR of the predicted frame with reference to the original frame The objective is to get a good improvement in the quality of SI frame 3 Results The half pel motion estimation is implemented for WZ frame generation using JM reference software The even frames are encoded as I frames and odd frames are encoded using WZ encoder For quality comparison between WZ encoder and H 264 encoder a separate encoding is done with even frame being I frame and odd frame being P frame The WZ frame obtained using SI prediction is analyzed for PSNR with reference to corresponding H 264 P frame The average PSNR plot for a QCIF 176x144 test sequence for a ME search range of 64 is shown in Table 1 and the plot is shown in Figure 6 Coastguard qcif yuv Search Range64 SI prediction scheme PSNR dB Frame Average 23 594586 Full pel ME 4x4 21 586695 Full pel ME 8x8 26 413514 Full pel ME 16x16 28 0283 Half pel ME 4x4 28 767624 Half pel ME 8x8 30 579922 Half pel ME 16x16 30 350176 Table 1 PSNR for WZ frame Figure 6 PSNR for WZ frame The PSNR plot in the Figure 6 indicates that the half pel ME with 8x8 block size performs better than any other schemes The gain in PSNR using ME for prediction is approximately 7dB compared that of
View Full Document
Unlocking...