Applying 3-D Methods to Video for CompressionOUTLINEPowerPoint PresentationSlide 43-D Transform MethodsPREVIOUS WORKALGORITHMS IN OUR EXPERIMENTS2-D versus 3-D DCT3D Volume Motion Compensated CodingSlide 10Initial Prediction Results3D Principal Component AnalysisExperimental SetupMiss AmSlide 15Salesman – 2D-MC vs 3D-MCSalesman – 3D-DCT vs 3D-MCForemanBusCONCLUSIONApplying 3-D Methods to Video for CompressionSalih Burak GokturkAnne Margot Fernandez AaronMarch 13, 2002EE 392J Project PresentationOUTLINE•MOTIVATION•PREVIOUS WORK•ALGORITHMS•RESULTS •CONCLUSIONBACKGROUND•Most common video compression scheme •Blockwise transform (i.e. 2D-DCT) for spatial compression•Blockwise Motion Estimation and Compensation for temporal redundancy•Used in MPEG•Motion Compensated Predictive Coding is better than Image based coding since it uses a good prediction from the previous frame.OBJECTIVE•How? Use 3-D Transformations and Methods. Exploit Temporal Redundancies MORE!•Why?3-D Transform Methods•Good for ideal case – smooth motion, global motion•Bad for non-uniform motion•Need large number of frame storesPREVIOUS WORK• 3-D DCT for Video–Bauer and Sayood, Chan and Sui, Servais and de Jager• 3-D Wavelets for Video– Pearlman et al. , Taubman et al., • Motion Compensated CodingALGORITHMS IN OUR EXPERIMENTS1. 2D DCT2. 3D DCT3. (Block) Motion Compensated Coding4. 3D (Volume) Motion Compensated Coding5. Other Transformation as a prediction scheme (PCA, Wavelet)2-D versus 3-D DCT•2-D Blocks versus 3-D Blocks•2-D DCT versus 3-D DCT•When would 3-D DCT not work?Markov-1 chainP(Xn=Xn-1) = pXn uniform [0, 255]3D Volume Motion Compensated Coding Current stack of framesPrevious stack of framesDivide stack into volume blocksFind 3D motion vectors For each volume block…Calculate residual3D DCT on residualQdepth3D Volume Motion Compensated Coding •Finding the motion vectors–Full Search–Measurement criterion•MAD•standard deviation of residual•When it does not work–No smooth motion (acceleration present)–Cases when 2D MC does not work (occlusion, illumination, non-uniform motion)–Block size (depth) along time axis is too small or too big• Centroid Effect : Center of the block usually has a better prediction than edges of the block.Initial Prediction Results2D MC Prediction 3D MC Prediction3D Principal Component Analysis Current stack of framesDivide each stack into volume blocksFind PCACoefs For each volume block…Calculate residual3D DCT on residualQ1 – Apply PCA on a large collection of blocks2 – Each Block is represented by a linear combination of principal components: =1+ 2+ 3+…= A = U  VT[ ] …U=[ ]…Experimental Setup•Techniques1. 2D DCT by 8x8 Blocks2. 3D DCT by 8x8x8 Blocks3. Block Motion Compensated Coding (8x8 and 16x16 followed by 2-D DCT)4. Volume Motion Compensated Coding (8x8x4, 8x8x6 and 8x8x8) followed by 3-D DCT.5. Principal Component Analysis (use principal components of 8x8x8 blocks as prediction followed by 3-D DCT) • Bit Rate vs PSNR Comparison•DCT coefficients were quantized using a uniform quantizer•PSNR was computed over the same support region.•Bit rate was calculated by finding the theoretical entropy of the quantized DCT coefficients and motion vectors.Miss AmMiss Am2D-DCT2D-MC 3D-DCT3D-PCA3D-MC0.14 bit/pixelSalesman – 2D-MC vs 3D-MC2D-MC3D-MC0.11 bit/pixelSalesman – 3D-DCT vs 3D-MC3D-DCT3D-MC0.16 bit/pixelForeman2D-MC3D-DCT3D-MC0.56 bit/pixelBusCONCLUSION•There is redundancy across temporal domain.•There is a lot of room for improvement if we use 3-D compression methods!•2-D motion compensated coding is good for movies with A LOT of motion.•When movements are smooth, 3-D methods will perform better than 2-D motion compensated coding.FUTURE WORK•Find Better Residual Coding Schemes for 3D MC. (That deals with centroid effect better)•Other 3D Methods