A Study on Structural Similarity Based Interframe Video Coding Att Kruafak att kruafak uta edu Project presentation EE5359 Multimedia Processing on July 29 2008 MULTIMEDIA PROCESSING LAB ELECTRICAL ENGINEERING DEPARTMENT THE UNIVERSITY OF TEXAS AT ARLINGTON 1 25 THE UNIVERSITY OF TEXAS AT ARLINGTON Contents Motivation Background on SSIM Measurement formulation Block matching with SAD and SSIM MULTIMEDIA PROCESSING LAB Codec implementation Codebook training Simulation results Conclusion and future work 2 25 Motivation Motivation Background on SSIM Measurement formulation Block matching with SAD SSIM The codec implement Codebook training Simulation results Conclusion future work The goal of the research is to study rate distortion behavior of the interframe codec designed with novel motion estimation based on structural similarity distortion and the codec with conventional motion estimation based on pixel error distortion absolute differences The study from previous literature shows that structural similarity metric provides better image assessment than pixel error based metric mean square error and peak signal to noise ratio The codec with fixed motion block sizes 16x16 8x8 and 4x4 will be investigated Simulation results such as rate distortions will be measured in terms of both peak signal to noise ratio versus bit rate and structural similarity index versus bit rate 3 25 Background on SSIM Motivation Measurement formulation Block matching with SAD SSIM The codec implement Codebook training Simulation results Conclusion future work SSIM x y l x y c x y s x y Background on SSIM l x y 1 x N 2 x y C1 x2 y2 C1 2 x y C2 x2 y2 C2 s x y Wang IP04 2 xy C3 x y C3 N x i 1 i 1 N 2 2 x xi x N 1 i 1 xy c x y 1 N xi x yi y N 1 i 1 x and y are two local window sizes N N xi is the i th pixel of x where i 1 N x is the mean of x x2 is the variance of x xy is the covariance of x and y Three properties of SSIM index symmetry boundedness unique max Wang IP04 SSIM in video codec design Mai ICASSP06 Yang 07 JVT AB31 4 25 RGB PSNR 27 6782 dB RGB SSIM 0 8229 Motivation 50 Background on SSIM 100 Measurement 150 formulation Block matching 200 with SAD SSIM 250 The codec 300 implement Codebook 350 training 400 Simulation results 450 Conclusion 500 future work 550 100 200 300 Original Football Fr 51 400 500 600 RGB SSIM map 700 800 900 1000 Reconstructed Fr 51 QP37 IPP RGB luminance comparison map RGB contrast comparison map RGB structure comparison map 5 25 Measurement formulation Motivation Background on SSIM Measurement formulation Block matching with SAD SSIM The codec implement Codebook training Simulation results Conclusion future work 1 Y SSIM F M 1 RGB SSIM F M BitRate in bps F M SSIM p 1 j 1 Y pj 1 1 1 R G B SSIM SSIM SSIM pj pj pj 3 3 p 1 j 1 3 F M TotalBit in bits F number of frames FrameRate in fps Cr SSIM Ypj SSIM Cb pj SSIM pj are the SSIM index values SSIM map of Y Cb and Cr components of the jth local window in the pth video frame F is the number of video frames M is the number of local windows SSIM Rpj SSIM Gpj SSIM Bpj are YCbCrto RGB conversion SSIM index values of R G and B components of the jth local window in the pth video frame 6 25 Block matching with SAD and SSIM Motivation Background on SSIM Measurement formulation Block matching with SAD SSIM The codec implement Codebook training Simulation results Conclusion future work N 1 N 1 SAD N D k l i j y k m l n p yRef k m i l n j p 1 n 0 m 0 DN k l min DNSAD k l i j i j mvN k l i1 j1 DNSSIM k l i j 1 SSIM DN k l min DNSSIM k l i j i j mvN k l i2 j2 7 25 The codec implement Motivation Background on SSIM Measurement formulation Block matching with SAD SSIM The codec implement Codebook training Simulation results Conclusion future work 8 25 1 4 pel interpolation on luma and 1 8 pel interpolation on chroma Motivation 50 Background on 60 SSIM 25 25 30 30 Measurement 70 formulation 35 35 40 Block matching with SAD 90 SSIM 160 The codec implement Cb Y 80 80 170 180 190 200 90 40 100 Cr 80 90 100 210 160 200 Codebook 220 training 160 180 180 200 240 200 Simulation 260 results 220 220 240 280 Conclusion 300 future work 240 260 260 280 320 280 300 340 300 320 360 320 340 650 700 750 800 650 700 750 800 650 700 Football CIF 4 2 0 reference frame 50 QP 30 stepsize 20 Interpolation method H 264 Richardson 03 9 25 750 800 Initial search point initial motion vector search Start Get sub block positions D B C X is a current sub block A X A B C D are neighboring sub block Macroblock k nc 1 Macroblock k nc Macroblock k nc 1 Sub block A exists No mvA 0 0 mvA Yes Macroblock k 1 4 2 4 3 1 3 4 2 3 Sub block B exists Sub blocks 8x8 1 to 4 No Replace mvB with mvA Yes mvB 4 Current macroblock k 16x16 Sub block C exists No Sub block D exists Yes Yes No Replace mvC with mvA Replace mvC with mvD mvC Macroblock k 1 mvPred median round mvA mvB mvC 4 Stop 10 25 15 y row index 10 Integer pel search 5 Fractional pel search 0 5 10 15 15 10 5 0 5 x column index 10 15 15 5 0 5 1 1 15 10 5 0 5 x column index 4 2 10 15 0 5 0 0 5 1 1 0 y row pixel index 15 y row pixel index 10 y row pixel index y row index 10 0 5 0 0 5 x column pixel index 1 0 5 0 0 5 1 1 0 5 0 0 5 x column pixel index 2 4 4 2 0 2 4 x column pixel index Fast motion estimation JVT Q89 JVT F17 JVT P21 JVT N46 Wedi 03 Hexagon based search and diamond based search Zhu 02 Zhu 00 11 25 1 Motivation Background on SSIM 100 200 Measurement formulation Block matching with SAD SSIM The codec implement Codebook training Simulation results 300 400 500 600 Conclusion future work 700 800 100 200 300 400 500 600 700 800 900 1000 Ref Fr 50 QP30 Motion predicted Fr 51 Original current Fr 51 16x16 motion block 12 25 Codebook training Motivation Background on SSIM Measurement formulation Fixed size integer transforms Block matching with SAD SSIM The codec implement Codebook training Histogram Huffman tree Huffman codebook Simulation results Conclusion future work Fixed size motion estimation 13 25 Table 3 2 Huffman codebook names and their descriptions …
View Full Document
Unlocking...