Advanced Video Compression StandardsOverviewFormal StandardsVideo Coding Standardization OrganizationsDynamics of the Video Standardization ProcessThe Scope of Picture and Video Coding StandardizationH.261: The Basis of Modern Video CompressionTypical MC+DCT Video CoderVideo Coding EfficiencyMPEG-1: Practicality at Higher Bit RatesMPEG-2/H.262: Even Higher Bit Rates and InterlaceH.263: The Next GenerationMPEG-4: Baseline H.263 and Many Creative ExtrasMPEG-4 and H.263 Standardization DynamicsDetailed Recent History In Video Coding StandardizationH.263++ New Version 3 Features Part 1 of 2H.263++ New Version 3 Features Part 2 of 2PowerPoint PresentationSlide 19MPEG-4 Version 3 Just Completed (part 1 of 2)MPEG-4 Version 3 Just Completed (part 2 of 2)ITU-T VCEG H.26L Project Goals (Completion 2002)H.26L StatusThe H.26L TML-6 Design Part 1 of 4The H.26L TML-6 Design Part 2 of 4The H.26L TML-6 Design Part 3 of 4The H.26L TML-6 Design Part 4 of 4Future Work in MPEGSlide 29Slide 30Slide 31Windows Media Technologies Video-Related FeaturesFuture TrendsPrinciples of Rate-Distortion TheoryOn Rate-Distortion OptimizationSome Future ProjectionsWhat Area will Yield the Most Improvement?Won’t This be Unnecessary when Megabits become free?Increasing “Layers” of StandardizationOther Kinds of LayersConclusionsAdvanced Video Compression StandardsGary Sullivan([email protected])Microsoft Corp. Software Design EngineerITU-T Rapporteur of Advanced Video CodingITU-T Recommendation H.263 EditorStanford, February 15, 2001Overview•Video Standardization Concepts•History•Recent events–Standardization projects–H.263 v1 & H.263+ & H.263++–MPEG-4 v1, v2, v3–H.26L Progress–Microsoft Windows Media Video•Future Stuff–H.26L Finalization–MPEG Video Plans–TrendsFormal Standards•Specification available to all at little or no cost•Anyone allowed to implement•Agreement officially by consensus, not decided by a single organization’s interests•Relatively open committee with variety of participants (including hostile competitors, with no contract to support a common agenda, often meeting with formal government approval)•In practice, each standards organization tends to have its own “personality”Video CodingStandardization Organizations•Two organizations dominate video compression standardization:–ITU-T Video Coding Experts Group (VCEG)International Telecommunications Union – Telecommunications Standardization Sector (ITU-T, a United Nations Organization, formerly CCITT), Study Group 16, Question 6–ISO/IEC Moving Picture Experts Group (MPEG)International Standardization Organization and International Electrotechnical Commission, Joint Technical Committee Number 1, Subcommittee 29, Working Group 11Dynamics of the VideoStandardization Process•VCEG is older and more focused on conventional (esp. low-delay) video coding goals (e.g. good compression and packet-loss/error resilience)•MPEG is larger and takes on more ambitious goals (e.g. “object oriented video”, “synthetic-natural hybrid coding”, and digital cinema)•Sometimes the major organizations team up (e.g. ISO, IEC and ITU teamed up for both MPEG-2 and JPEG)•Relatively little industry consortium activity (DV and organizations that tweak the video coding standards in minor ways, such as DVD, 3GPP, 3GPP2, SMPTE, IETF, etc.)•Growing activity for internet streaming media outside of formal standardization (e.g., Microsoft, Real Networks, Quicktime)The Scope of Picture and Video Coding Standardization•Only the Syntax and Decoder are standardized:–Permits optimization beyond the obvious–Permits complexity reduction for implementability–Provides no quality guarantees – only interoperabilityPre-Processing EncodingSourceDestinationPost-Processing DecodingScope of StandardH.261: The Basis of Modern Video Compression•ITU-T (ex-CCITT) Rec. H.261: The first widespread practical success–First design (late ‘90) embodying typical structure that dominates today: 16x16 macroblock motion compensation, 8x8 DCT, scalar quantization, and variable-length coding–Key aspects later dropped by other standards: loop filter, integer motion comp., 2-D VLC, header overhead–v2 (early ‘93) added a backward-compatible high-resolution graphics trick mode–Operated at 64-2048 kbps–Still in use, although mostly as a backward-compatibility feature – overtaken by H.263Entropy Decode,Quant. Recon.,Inverse DCTTypical MC+DCT Video CoderMotion Comp.PredictorDCT,Quantize,Entropy EncodeMotionEstimationFrame Buffer(Delay)MotionCompensatedPredictionInputFrameEncoded Residual (To Channel)ApproximatedInput Frame (To Display)Motion Vector andBlock Mode Data (To Channel)Prior CodedFrame Approx(Dotted BoxShows Decoder)Video Coding Efficiency0 100 200 300 400 50026283032343638Foreman10 Hz, QCIF100 frames encodedInteger-pelmotioncompensation(H.261 1991)Half-pelmotioncompensation(MPEG-1 1993)TMN-10Variableblock sizemotioncompensation(H.263 1998)Framedifferencecoding(H.120 1988)IntraframeDCT coding(DCT 1974, JPEG 1992)PSNR[dB]Bit-Rate [kbps]?67 %MPEG-1:Practicality at Higher Bit Rates•Formally ISO/IEC 11172-2 (‘93), developed by ISO/IEC JTC1 SC29 WG11 (MPEG) – use is fairly widespread, but mostly overtaken by MPEG-2–Superior quality to H.261 when operated a higher bit rates ( 1 Mbps for CIF 352x288 resolution)–Can provide approximately VHS quality between 1-2 Mbps using SIF 352x240/288 resolution–Technical features: Adds bi-directional motion prediction and half-pixel motion to H.261 designMPEG-2/H.262: Even Higher Bit Rates and Interlace•Formally ISO/IEC 13818-2 & ITU-T H.262, developed (‘94) jointly by ITU-T and ISO/IEC SC29 WG11 (MPEG) – Now in wide use for DVD and standard and high-definition DTV (the most commonly used video coding standard)–Primary new technical features: support for interlaced-scan pictures and scalability–Essentially the same as MPEG-1 for progressive-scan pictures, and MPEG-1 forward compatibility required–Not especially useful below 4 Mbps (range of use normally 5-30 Mbps)H.263: The Next Generation•ITU-T Rec. H.263 (v1: 1995): The next generation of video coding performance, developed by ITU-T – the current best standard for practical video telecommunication (has overtaken H.261 as dominant videoconferencing codec)–Superior to H.261 at all bit rates–Wins by a factor of two at very low rates–Versions 2 (late 1997/early 1998) & v3 (2000) later developedMPEG-4: