Audio coding and systemsH.264/MPEG-4 Part 10 VIDEO CODING FOR NEXT GENERATION MULTIMEDIAThe video coding standards developed earlier by ISO/IEC and ITU-T have not been able to address all the needs required by varying bit rates of different applications and at the same time meeting the quality requirements. A recently established video coding standard named H.264/MPEG-4 part 10 (initial version became International standardin July 2003) aims at coding video sequences at . approximately half the bit rate compared to MPEG-2 at the same quality.( See A. Puri ,H. Chen and A. Luthra, "Video Coding using the H.264/MPEG-4 AVC compression standard", Signal Processing: Image Communication vol. 19, pp.793-849, Oct. 2004. (www.sciencedirect.com)). Several improvements, additions, enhancements in terms of versions, profiles etc have since been developed. This, of course, is an ongoing process. It also aims at having significant improvements in coding efficiency, error robustness and network friendliness. It makes use of better prediction methods for Intra (I), Predictive (P) and Bi-predictive (B) frames. Arbitrary Block-size Transform (ABT) is used which is a simplified transform that avoids the mismatch error (DCT/IDCT) observed in the motion compensation hybrid coding adopted in MPEG-1 and MPEG-2. All these features along with others such as CABAC (Context Based Adaptive Binary Arithmetic Coding) have resulted in having a 2:1 coding gain over MPEG-2 atthe cost of increased complexity. This emerging standard addresses various services/applications, transmission networks, enhanced efficiency, and diverse range of bit rates and spatial/temporal resolutions through profiles and layers. Parameter setconcept, arbitrary slice ordering, flexible macroblock structure, redundant pictures, switched predictive and switched intra pictures have contributed to error resilience/robustness of this standard. Adaptive (directional) intra prediction, multiple reference pictures/frames for motion estimation and weighted motion compensated (MC), variable block-size MC, deblocking filter, hierarchical block transform etc., have contributed to the high coding efficiency of this standard developed jointly by the ITU-T Video coding Experts Group (VCEG) and the IS0/IEC Moving Picture Experts Group (MPEG). This combined group is called JointVideo Team (JVT)..Other parts of this standard such as file format, verification testing, reference software, conformance bit streams, standardizing example encodingdescription, scalable video coding and potential extensions have been almost finalized. The tutorial highlights the various functionalities of the encoder, points out the differences between this new standard and the existing standards and describes thestate-of-the-art development by the industry. ftp and web sites related to standard documents, software, databases, conformance bit streams, meeting schedules, vendors, file formats, research groups, FAQ etc are provided. This standard opens up several research areas based on software/hardware implementations, improvements etc. For example transcoding between H.264 and MPEG-2 (various levels and profiles) is can provide forward/backward compatibilities.The new standard addresses various applications such as video streaming over the internet, conversational services such as videophone/video conferencing over wired and wireless (mobile) networks, video-on-demand, near video-on-demand, multimedia messaging, pay-per-view, digital TV, HDTV, super HDTV, digital cinema, high quality video transmission over cable, cable modem, DSL, satellite and terrestrial channels, high density optical storage media such as DVD/HDDVD, digital cameras, camcorders and related consumer electronicelectronics products. Thisemerging/established standard enables multimedia services/systems viable, feasible, practical, affordable and user friendly. Through various network layers/protocols, profiles and levels this standard is designed to meet the ever increasing needs of current and emerging multimedia applications. The innovative and ingenious approach adopted in this multidimensional signal processing has led to 2:1 bit rate reduction while maintaining the same high visual quality as MPEG-2. Several start-ups as well as established/reputed multinational companies, research institutes etc have embarked on ambitious projects for developing/marketing consumer oriented products based on this standard. Many products/devices have already come into the market. It is projected that this will be a multibillion dollar market within the next few years. Further developments under fidelity range extensions (FRExt, Amendment 1, approved in July 2004) are:◈ Extension to 4:2:2 and 4:4:4 chroma formats◈ 9 and 10 bit resolutions◈ 11 and 12 bit resolutions ◈ Scalable coding, Lossless coding for digital cinema application◈ High fidelity coding for the next generation optical discs.8x8 and 4x4 adaptive integer DCT◈ source editing functions such as alpha blending◈ Encoder specified HVS weighted quantization◈ Lossless coding of specific regions in video content◈ Separate Cb and Cr QP control◈ Residual color transform for 4:4:4 formatFRExt. has led to four new profiles called High profiles. These are seriously being considered/adopted for application specifications such as HD-DVD specification by the DVD Forum (now adopted)BD-ROM Video specification by the Blu-Ray Disc associationDVB standards for European broadcast television (Digital Video Broadcasting)ATSC (US) Advanced Television Systems CommitteeVarious designs for satellite and cable TVSee paper G. J. Sullivan, P. Topiwala and A. Luthra “The H.264/AVC advanced video coding standard: Overview and introduction to the fidelity range extensions”, SPIE Conf. onapplications of digital image processing XXVII, Special session on advances in the new emerging standard: H.264/AVC., vol. 5558, pp. 53-74, Aug. 2004, The authors claim thatwith these extensions, H.264 can achieve a 3:1 coding gain over MPEG-2 in some key applications. There are many other papers on H.264 in this special session. See G.J. Sullivan, “ The H.264/MPEG-4 AVC video coding standard and its deployment status”, SPIE/VCIP 2005, Vol. 5960, pp. 709-719, Beijing, China, July 2005. This paper summarizes deployment of H.264 based devices by Industry and various organizations. See alsoH.264/MPEG-4 AVC Application & Deployment Status – Thomas Wiegand (Assoc.