Video Coding StandardsYao WangPolytechnic University, Brooklyn, [email protected]©Yao Wang, 2003 EE4414: Video Coding Standards 2Outline• Overview of Standards and Their Applications• ITU-T Standards for Audio-Visual Communications–H.261–H.263– H.263+, H.263++, H.264• ISO Standards for –MPEG-1–MPEG-2–MPEG-4–MPEG-7©Yao Wang, 2003 EE4414: Video Coding Standards 3Multimedia Communications Standards and ApplicationsStandards Application Video Format Raw Data Rate Compressed Data Rate H.320 (H.261) Video conferencing over ISDN CIF QCIF 37 Mbps 9.1 Mbps >=384 Kbps >=64 Kbps H.323 (H.263) Video conferencing over Internet 4CIF/ CIF/ QCIF >=64 Kbps H.324 (H.263) Video over phone lines/ wireless QCIF 9.1 Mbps >=18 Kbps MPEG-1 Video distribution on CD/ WWW CIF 30 Mbps 1.5 Mbps MPEG-2 Video distribution on DVD / digital TV CCIR601 4:2:0 128 Mbps 3-10 Mbps MPEG-4 Multimedia distribution over Inter/Intra net QCIF/CIF 28-1024 Kbps GA-HDTV HDTV broadcasting SMPTE296/295 <=700 Mbps 18--45 Mbps MPEG-7 Multimedia databases (content description and retrieval)©Yao Wang, 2003 EE4414: Video Coding Standards 4H.261 Video Coding Standard• For video-conferencing/video phone– Video coding standard in H.320– Low delay (real-time, interactive)– Slow motion in general• For transmission over ISDN– Fixed bandwidth: px64 Kbps, p=1,2,…,30• Video Format: – CIF (352x288, above 128 Kbps) – QCIF (176x144, 64-128 Kbps)– 4:2:0 color format, progressive scan• Published in 1990• Each macroblock can be coded in intra- or inter-mode• Periodic insertion of intra-mode to eliminate error propagation due to network impairments• Integer-pel accuracy motion estimation in inter-mode©Yao Wang, 2003 EE4414: Video Coding Standards 5H.261 EncoderF: Loop filter; P: motion estimation and compensationLoop filter: apply low-pass filter to smooth the quantization noise in previously reconstructed frames before motion estimation and compensation©Yao Wang, 2003 EE4414: Video Coding Standards 6DCT Coefficient QuantizationDC Coefficient in Intra-mode:Uniform, stepsize=8Others:Uniform with deadzone, stepsize=2~64 (MQUANT)Deadzone:To avoid too many small coefficients being coded, which are typically due to noise©Yao Wang, 2003 EE4414: Video Coding Standards 7Motion Estimation and Compensation• Integer-pel accuracy in the range [-16,16]• Methods for generating the MVs are not specified in the standard – Standards only define the bitstream syntax, or the decoder operation)• MVs coded differentially (DMV)• Encoder and decoder uses the decoded MVs to perform motion compensation • Loop-filtering can be applied to suppress propagation of coding noise temporally– Separable filter [1/4,1/2,1/4]: applied horizontally and vertically– Loop filter can be turned on or off©Yao Wang, 2003 EE4414: Video Coding Standards 8Parameter Selection and Rate Control• MTYPE (intra vs. inter, zero vs. non-zero MV in inter)• CBP (which blocks in a MB have non-zero DCT coefficients)• MQUANT (allow the changes of the quantizer stepsize at the MB level)– should be varied to satisfy the rate constraint• MV (ideally should be determined not only by prediction error but also the total bits used for coding MV and DCT coefficients of prediction error)• Loop Filter on/off©Yao Wang, 2003 EE4414: Video Coding Standards 9Variable Length Coding • DCT coefficients are converted into runlength representations and then coded using VLC (Huffman coding for each pair of symbols)– Symbol: (Zero run-length, non-zero value range)• Other information are also coded using VLC (Huffman coding)– MTYPE–CBP–MQUANT–MV– Loop filter on/off©Yao Wang, 2003 EE4414: Video Coding Standards 10H.263 Video Coding Standard• H.263 is the video coding standard in H.323/H.324, targeted for visual telephone over PSTN or Internet• Developed later than H.261, can accommodate computationally more intensive options– Initial version (H.263 baseline): 1995– H.263+: 1997– H.263++: 2000• Goal: Improved quality at lower rates• Result: Significantly better quality at lower rates– Better video at 18-24 Kbps than H.261 at 64 Kbps– Enable video phone over regular phone lines (28.8 Kbps) or wireless modem©Yao Wang, 2003 EE4414: Video Coding Standards 11Improvements over H.261• Better motion estimation – half-pel accuracy motion estimation with bilinear interpolation filter – Larger motion search range [-31.5,31], and unrestricted MV at boundary blocks– More efficient predictive coding for MVs (median prediction using three neighbors)– overlapping block motion compensation (option)– variable block size: 16x16 -> 8x8, 4 MVs per MB (option)– use bidirectional temporal prediction (PB picture) (option)• 3-D VLC for DCT coefficients – (runlength, value, EOB)• Syntax-based arithmetic coding (option)– 4% savings in bit rate for P-mode, 10% saving for I-mode, at 50% more computations• The options, when chosen properly, can improve the PSNR 0.5-1.5 dB over default at 20-70 kbps range.©Yao Wang, 2003 EE4414: Video Coding Standards 12Performance of H.261 and H.263Forman, QCIF, 12.5 HzInteger MC, +/- 16Half-pel MC, +/- 32Integer MC, +/- 16, loop filterInteger MC, +/- 32OBMC, 4 MVs, etc©Yao Wang, 2003 EE4414: Video Coding Standards 13ITU-T Multimedia Communications Standards©Yao Wang, 2003 EE4414: Video Coding Standards 14H.324 Terminal(multimedia communication over PSTN)©Yao Wang, 2003 EE4414: Video Coding Standards 15MPEG-1 Overview• Audio/video on CD-ROM (1.5 Mbps, CIF: 352x240)– Maximum: 1.856 mbps, 768x576 pels• Start late 1988, test in 10/89, Committee Draft 9/90 • ISO/IEC 11172-1~5 (Systems, video, audio, compliance, software).• Prompted explosion of digital video applications: MPEG1 video CD and downloadable video over Internet• Software only decoding, made possible by the introduction of Pentium chips, key to the success in the commercial market• MPEG-1 Audio– Offers 3 coding options (3 layers), higher layer have higher coding efficiency with more computations– MP3 = MPEG1 layer 3 audio©Yao Wang, 2003 EE4414: Video Coding Standards 16MPEG-1 video vs H.261• Developed at about the same time• Must enable random access (Fast forward/rewind)– Using GOP structure with periodic I-picture and P-picture• Not for interactive applications– Do not have as stringent delay requirement• Fixed rate (1.5 Mbps), good quality (VHS