New Techniques for Improved Video Coding Thomas Wiegand Fraunhofer Institute for Telecommunications Heinrich Hertz Institute Berlin Germany wiegand hhi de Outline Inter frame Encoder Optimization Texture Analysis and Synthesis Model Aided Coding T Wiegand New Techniques for Improved Video Coding 1 Scope of Video Coding Standardization Only Restrictions on the Bitstream Syntax and Decoder are standardized Permits optimization beyond the obvious Permits complexity reduction for implementability Provides no guarantees of quality Source Sink Pre Processing Encoding Post Processing Error Recovery Decoding Scope of Standard T Wiegand New Techniques for Improved Video Coding 2 Hybrid Video Encoder Input Video Signal Coder Control Split into Macroblocks 16x16 pixels Control Data Transform Scal Quant Decoder Quant Transf coeffs Scaling Inv Transform Entropy Coding Intra frame Prediction De blocking Filter MotionIntra Inter Compensation Output Video Signal Motion Data Motion Estimation T Wiegand New Techniques for Improved Video Coding 3 Inter frame Optimization of Video Encoding Most video encoders select locally optimal transform coefficients as they ignore temporal dependencies Time Motion Compensation New approach that takes inter frame dependencies into account T Wiegand New Techniques for Improved Video Coding 4 Linear Signal Model for Decoder motion data transform coefficients s Ms Tc p reconstructed sample values static predictor inverse transform scaling Very large size of matrices Numbers when optimizing 4 QCIF pictures jointly M T 100 000 x 100 000 entries T Wiegand New Techniques for Improved Video Coding 5 Quadratic Program Use Mean Squared Error distortion Select transform coefficient levels by solving original video signal minimize subject to s v s v abs c i T s Ms Tc p i Equivalent to a quadratic program Can be solved by standard solution methods T Wiegand New Techniques for Improved Video Coding 6 Results Used H 264 MPEG4 AVC Compare looking at 2 or 3 frames at a time Compare to H 264 MPEG4 AVC Test Model Constant QP Motion estimation on original frames Only optimized inter frames T Wiegand New Techniques for Improved Video Coding 7 FlowerGarden 30Hz IPPP Average Y PSNR dB 39 38 37 36 35 34 33 32 TM2 2 Frames Jointly 3 Frames Jointly 31 30 29 0 100 200 300 Bit Rate kbits sec 400 500 16 Bit rate improvement T Wiegand New Techniques for Improved Video Coding 8 FlowerGarden 30Hz IbPbPbP 37 Average Y PSNR dB 36 35 34 33 32 TM2 3 Frames Jointly 31 30 0 50 100 150 200 Bit Rate kbits sec 250 300 15 Bit rate improvement T Wiegand New Techniques for Improved Video Coding 9 Summarizing Inter frame Optimization Very complex has only recently become possible Problems with H 264 MPEG4 AVC rounding Adjust design to support efficient encoding methods Heavy incorporation of encoding methods has been used in H 264 MPEG4 AVC development with Lagrangian encoder optimization Do not standardize the encoder T Wiegand New Techniques for Improved Video Coding 10 Texture Analysis and Synthesis Textures with large amount of visible details are difficult to code e g grass sand clouds water Viewer often does not perceive large differences between different versions of grass sand clouds water Where is the original texture T Wiegand New Techniques for Improved Video Coding 11 Video Analysis and Synthesis Approach Exact reproduction of details is irrelevant if Textures are shown with limited spatial accuracy Viewer does not know the original video Consequence Mean Squared Error MSE distortion criterion is not suitable for efficient coding of detail irrelevant textures Issue Could we use similarity criteria color histograms etc instead of MSE as coding distortion Scheme If information needed for approximate reproduction of detail irrelevant textures requires less bit rate than using MSE bit rate savings T Wiegand New Techniques for Improved Video Coding 12 Video Codec Architecture Video In Encoder TA Bits Side Info Decoder Video Out TS Video coding using a texture analyzer TA and a texture synthesizer TS T Wiegand New Techniques for Improved Video Coding 13 Texture Analyzer Principle Ref mask Ref frame Split Merge Segmentation Motion Compensation Texture Analyzer Current mask Current frame T Wiegand New Techniques for Improved Video Coding 14 Texture Synthesizer Stuffing procedure T Wiegand New Techniques for Improved Video Coding 15 Texture Synthesizer Match surrounding samples between current picture and warped pictures to find synthesis sample for the current sample Decoder processing required Volume matching for water and temporal consistence T Wiegand New Techniques for Improved Video Coding 16 Flowergarden CIF 30 Hz QP 16 41 frames Bit rate savings 19 4 T Wiegand New Techniques for Improved Video Coding 17 Canoe CIF 30 Hz QP 16 73 frames Bit rate savings 8 8 T Wiegand New Techniques for Improved Video Coding 18 Summarizing Texture Analysis and Synthesis Textures with large amount of visible details are difficult to code e g grass sand clouds water Errors in these textures are barely visible Texture analysis and synthesis can help to represent these textures efficiently Method requires processing at the decoder side Other methods require processing at the decoder side to achieve coding efficiency improvements through backward adaptation CABAC T Wiegand New Techniques for Improved Video Coding 19 Model Aided Coding Model based coding MBC Extreme low bit rates possible Missing generality Hybrid video coding H 263 4 MPEG 2 4 Robust against scene changes Higher bit rates Combination of MBC with hybrid video coding Model aided codec High coding efficiency No restriction to scene content T Wiegand New Techniques for Improved Video Coding 20 Architecture of the Model Aided Codec video signal S x y t e CoderControl control data Intraframe DCT Coder DCTcoefficients Modelbased Coder Intraframe Decoder Motion Compensation Model based Decoder Model Frame Reconst Frame motion data FAPs Decoder T Wiegand New Techniques for Improved Video Coding 21 Coding Results Sequenz Clapper Board 8 33 Hz CIF resolution 260 frames Same average bit rate 12 kbit s H 263 annexes D F I J T H 263 TMN 10 Model Aided Codec MAC T Wiegand New Techniques for Improved Video Coding 22 Reconstruction Quality vs Bit rate 3 5 dB 45 Gain of 3 5 dB PSNR compared to H 263 Coder TMN 10 Bit rate reduction 45 T Wiegand New Techniques for Improved Video Coding 23 Model Aided Coding with Approximate Geometry H 264 TML 8 MAC T Wiegand New
View Full Document
Unlocking...