EE 5359 Low Complexity H.264 encoder for mobile applications Thejaswini Purushotham Student I.D.: 1000-616 811 Date: February 18,2010Objective The objective of the project is to implement a low-complexity encoder for mobile applications using machine learning algorithm. Motivation H.264 is currently one of the most widely accepted video coding standards in the industry. It enables high quality video at very low bitrates.H.264 is a block-oriented motion-compensation based video codec developed by the ITU-T Video Coding Experts Group(VCEG) together with the ISO/IEC Moving Pictures Expert Group(MPEG). H.264 has a highly complex encoder which leads to a good performance in terms of bit-rate. The high-computational complexity of H.264 and real-time requirements of video systems represent the main challenge to overcome the development of efficient encoder solutions. The computational complexity in H.264 comes from the motion estimation and mode decision techniques implemented in the encoder. Hence by reducing the computation complexity of the motion estimation and mode decision techniques, it will be possible to confide with the real-time requirements of the video systems. Details Overview of H.264: H.264 [1] is a standard for video compression, and is equivalent to MPEG-4 Part 10, or MPEG-4 AVC (for advanced video coding) (Fig. 2 and 3). As of 2008, it is the latest block-oriented motion-compensation-based video standard developed by the ITU-T Video coding experts group (VCEG) together with the ISO/IEC moving picture experts group (MPEG), and it was the product of a partnership effort known as the joint video team (JVT). The ITU-T H.264 standard and the ISO/IEC MPEG-4 part 10 standard (formally, ISO/IEC 14496-10) are jointly maintained so that they have identical technical content.Fig 1: Basic coding structure for H.264 /AVC for a macroblock [1] Fig. 2: H.264 Encoder [2] Fig. 3: H.264 Decoder [2] The standardization of the first version of H.264/AVC was completed in May 2003. The JVT then developed extensions to the original standard that are known as the fidelity range extensions (FRExt)[3]. These extensions enable higher quality video coding by supporting increased sample bit depth precision and higher-resolution color information, including sampling structures known as YUV 4:2:2 and YUV 4:4:4. Several other features are also included in the fidelity range extensions, such as adaptive switching between 4×4 and 8×8 integer transforms, encoder-specified perceptual-based quantization weighting matrices, efficient inter-picture lossless coding, and support of additional color spaces. The design work on the fidelity range extensions was completed in July 2004, and the drafting work on them was completed in September 2004. Scalable video coding (SVC) [4] as specified in Annex G of H.264/AVC allows the construction of bitstreams that contain sub-bitstreams that conform to H.264/AVC. For temporal bitstream scalability, i.e., the presence of a sub-bitstream with a smaller temporal sampling rate than the bitstream, complete access units are removed from the bitstream when deriving the sub-bitstream. In this case, high-level syntax and inter prediction reference pictures in the bitstream are constructed accordingly. For spatial and quality bitstream scalabilities, i.e. the presence of a sub-bitstream with lower spatial resolution or quality than the bitstream, network abstraction layer (NAL) units are removed from the bitstream when deriving the sub-bitstream. In this case, inter-layer prediction, i.e., the prediction of the higher spatial resolution or quality signal by data of the lower spatial resolution or quality signal, is typically used for efficient coding. The scalable video coding extension was completed in November 2007[4]. Some of the features adopted in H.264 for enhancement of prediction, improved coding efficiency and robustness to data errors/losses are listed as follows. Features for enhancement of prediction are as follows. Directional spatial prediction for intra coding Variable block-size motion compensation with small block size (Fig. 4) Fig. 4 :Various block sizes in H.264 for motion estimation/compensation [1]Quarter-sample-accurate motion compensation Motion vectors over picture boundaries Multiple reference picture motion compensation Decoupling of referencing order from display order Decoupling of picture representation methods from picture referencing capability Weighted prediction Improved “skipped” and “direct” motion inference In-the-loop deblocking filtering Features for improved coding efficiency are as follows. Small block-size transform Exact-match inverse transform (Fig. 5) Fig. 5:Forward 4x4 and 8x8 integer transforms [3] Short word-length transform Hierarchical block transform Arithmetic entropy coding Context-adaptive entropy codingFeatures for robustness to data errors/losses are as follows. Parameter set structure NAL unit syntax structure Flexible slice size Flexible macroblock ordering (FMO) Arbitrary slice ordering (ASO) Redundant slices (RS) Data partitioning SP/SI synchronization/switching pictures Profiles in H.264 H.264 standard defines numerous profiles, as listed below. Constrained baseline profile Baseline profile Main profile Extended profile High profile High 10 profile High 4:2:2 profile High 4:4:4 predictive profile High stereo profile High 10 intra profile High 4:2:2 intra profile High 4:4:4 intra profile CAVLC 4:4:4 intra profile Scalable baseline profile Scalable high profile Scalable high intra profile Table 1 and Table 2 outlines the features of the various profiles in H.264. Fig. 6 gives a graphical comparison of the profiles in H.264.Table 1 :Features in baseline, main and extended profile [3] Table 2 :Features in high profile [3]Fig. 6: Comparison of H.264 baseline, main, extended and high profiles [2] MACHINE LEARNING Machine learning usually refers to the changes in systems that perform tasks associated with the artificial intelligence (AI).Such tasks involve recognition, diagnosis, planning, robot control, prediction etc. The “changes ” might be either enhancements to already performing systems or ab initio synthesis of new systems. It is possible that hidden among large piles of data are important
View Full Document
Unlocking...