EE 5359 Low Complexity H 264 encoder for mobile applications Thejaswini Purushotham Student I D 1000 616 811 Date February 18 2010 Objective The objective of the project is to implement a low complexity encoder for mobile applications using machine learning algorithm Motivation H 264 is currently one of the most widely accepted video coding standards in the industry It enables high quality video at very low bitrates H 264 is a block oriented motion compensation based video codec developed by the ITU T Video Coding Experts Group VCEG together with the ISO IEC Moving Pictures Expert Group MPEG H 264 has a highly complex encoder which leads to a good performance in terms of bit rate The high computational complexity of H 264 and real time requirements of video systems represent the main challenge to overcome the development of efficient encoder solutions The computational complexity in H 264 comes from the motion estimation and mode decision techniques implemented in the encoder Hence by reducing the computation complexity of the motion estimation and mode decision techniques it will be possible to confide with the real time requirements of the video systems Details Overview of H 264 H 264 1 is a standard for video compression and is equivalent to MPEG 4 Part 10 or MPEG4 AVC for advanced video coding Fig 2 and 3 As of 2008 it is the latest block oriented motioncompensation based video standard developed by the ITU T Video coding experts group VCEG together with the ISO IEC moving picture experts group MPEG and it was the product of a partnership effort known as the joint video team JVT The ITU T H 264 standard and the ISO IEC MPEG 4 part 10 standard formally ISO IEC 14496 10 are jointly maintained so that they have identical technical content Fig 1 Basic coding structure for H 264 AVC for a macroblock 1 Fig 2 H 264 Encoder 2 Fig 3 H 264 Decoder 2 The standardization of the first version of H 264 AVC was completed in May 2003 The JVT then developed extensions to the original standard that are known as the fidelity range extensions FRExt 3 These extensions enable higher quality video coding by supporting increased sample bit depth precision and higher resolution color information including sampling structures known as YUV 4 2 2 and YUV 4 4 4 Several other features are also included in the fidelity range extensions such as adaptive switching between 4 4 and 8 8 integer transforms encoder specified perceptual based quantization weighting matrices efficient inter picture lossless coding and support of additional color spaces The design work on the fidelity range extensions was completed in July 2004 and the drafting work on them was completed in September 2004 Scalable video coding SVC 4 as specified in Annex G of H 264 AVC allows the construction of bitstreams that contain sub bitstreams that conform to H 264 AVC For temporal bitstream scalability i e the presence of a sub bitstream with a smaller temporal sampling rate than the bitstream complete access units are removed from the bitstream when deriving the sub bitstream In this case high level syntax and inter prediction reference pictures in the bitstream are constructed accordingly For spatial and quality bitstream scalabilities i e the presence of a sub bitstream with lower spatial resolution or quality than the bitstream network abstraction layer NAL units are removed from the bitstream when deriving the sub bitstream In this case inter layer prediction i e the prediction of the higher spatial resolution or quality signal by data of the lower spatial resolution or quality signal is typically used for efficient coding The scalable video coding extension was completed in November 2007 4 Some of the features adopted in H 264 for enhancement of prediction improved coding efficiency and robustness to data errors losses are listed as follows Features for enhancement of prediction are as follows Directional spatial prediction for intra coding Variable block size motion compensation with small block size Fig 4 Fig 4 Various block sizes in H 264 for motion estimation compensation 1 Quarter sample accurate motion compensation Motion vectors over picture boundaries Multiple reference picture motion compensation Decoupling of referencing order from display order Decoupling of picture representation methods from picture referencing capability Weighted prediction Improved skipped and direct motion inference In the loop deblocking filtering Features for improved coding efficiency are as follows Small block size transform Exact match inverse transform Fig 5 Fig 5 Forward 4x4 and 8x8 integer transforms 3 Short word length transform Hierarchical block transform Arithmetic entropy coding Context adaptive entropy coding Features for robustness to data errors losses are as follows Parameter set structure NAL unit syntax structure Flexible slice size Flexible macroblock ordering FMO Arbitrary slice ordering ASO Redundant slices RS Data partitioning SP SI synchronization switching pictures Profiles in H 264 H 264 standard defines numerous profiles as listed below Constrained baseline profile Baseline profile Main profile Extended profile High profile High 10 profile High 4 2 2 profile High 4 4 4 predictive profile High stereo profile High 10 intra profile High 4 2 2 intra profile High 4 4 4 intra profile CAVLC 4 4 4 intra profile Scalable baseline profile Scalable high profile Scalable high intra profile Table 1 and Table 2 outlines the features of the various profiles in H 264 Fig 6 gives a graphical comparison of the profiles in H 264 Table 1 Features in baseline main and extended profile 3 Table 2 Features in high profile 3 Fig 6 Comparison of H 264 baseline main extended and high profiles 2 MACHINE LEARNING Machine learning usually refers to the changes in systems that perform tasks associated with the artificial intelligence AI Such tasks involve recognition diagnosis planning robot control prediction etc The changes might be either enhancements to already performing systems or ab initio synthesis of new systems It is possible that hidden among large piles of data are important relationships and correlations Machine learning methods can often be used to extract the relationships data mining 11 The idea is to approximate a function or an unknown value using the statistics of the known data Statistical methods for dealing with these problems can be considered instances of machine learning because the decision and the estimation rules depend on a
View Full Document
Unlocking...