EE 5359 LOW COMPLEXITY H 264 ENCODER USING MACHINE LEARNING Presented by Thejaswini Purushotham 1000616811 1 ABSTRACT Abstract The macroblock mode decision in inter frames is computationally the most expensive process due to the use of features such as variable block size motion estimation and quarter pixel motion compensation in H 264 encoder Hence the goal of this project is to reduce the encoding time while conserving the quality and compression ratio Machine learning has been used to decide the mode decisions and hence reduce the motion estimation time The proposed machine learning method on an average decreases the encoding time by 42 864 for mode decisions in H 264 encoder and 01 decrease in SSIM 2 Table of Contents Page Abstract 2 List of Illustrations 4 List of Tables 5 Acronyms and Abbreviations 6 Overview of H 264 codec 8 H 264 AVC Profiles 10 H 264 Encoder and Decoder 15 Inter Prediction and Intra Prediction 18 WEKA 22 Test Sequences 25 JM H 264 27 Machine learning 28 Quality metrics 30 Experimental Results 34 Conclusion 45 References 46 3 LIST OF ILLUSTRATIONS Figures Page 1 Various profiles of H 264 14 2 Encoder and decoder block diagrams 15 3 A Multi frame motion compensation in H 264 18 B Multi frame bidirectional motion compensation in H 264 19 4 A Inter prediction in H 264 20 B Variable block sizes used for motion estimation and motion compensation 20 5 A Intra 4 4 prediction modes and prediction directions 21 B Intra 16 16 prediction modes and prediction directions 21 6 Intra 4x4 prediction for samples a p of a block 22 7 Supporting picture format 4 2 0 chroma sampling for CIF 25 8 Supporting picture format 4 2 0 chroma sampling for QCIF 26 9 Flow chart of machine learning process 28 10 Structural Similarity Index metric system 31 11 Classification tree for Container sequence from Weka tool 35 12 Tree stucture for Container sequence from Weka tool 36 13 Snap shot of the implementation of the Tree structure in JM 16 2 encoder 36 14 Snapshots of video sequences used 41 4 LIST OF TABLES Table Page 1 H 264 data rates at various resolutions 9 2 H 264 profiles for particular applications 11 3 H 264 MPEG 4 Part 10 profile specifications 12 4 Basic information for CIF sequence football yuv 26 5 Basic information for QCIF sequence Foreman yuv 26 6 Weka results 35 7 Results obtained using Normal JM and JM using machine learning 37 8 Speedup in Encoding time and ME time 39 9 Comparison of Compressed file size 40 10 Comparison of PSNR and MSE 42 11 SSIM comparison 44 5 ACRONYMS AND ABBREVIATIONS 3G Third generation 3GPP 3rd Generation Partnership Project AVC Advanced video coding ASO Arbitrary slice order B slice Bi predictive slice BP Baseline Profile CBP Constrained Baseline Profile CAVLC context adaptive variable length coding CABAC context adaptive binary arithmetic coding CIF common intermediate format DVB Digital Video Broadcasters DVD Digital Video Disc Fps Frames per sec FMO Flexible Macroblock Order HD High Definition HiP High Profile Hi10P High 10 Profile Hi422P High 4 2 2 Profile Hi444PP High 4 4 4 Predictive Profile iChat Instant chatting application ID3 Iterative Dichotomiser 3 I slice Intra slice JM Joint model MB Macroblock MP Main profile ME Motion estimation 6 MC Motion compensation MPEG Moving Picture Experts Group MSE Mean square error NAL Network abstraction layer P slice Predictive slice P Predicted macroblock PSNR Peak to peak signal to noise ratio QCIF quarter common intermediate format RS Redundant slice SI slice Switching I slice SP slice Switching P slice SSIM Structural similarity index metric VCEG Video Coding Experts Group XP Extended profile 7 Overview of H 264 Video codec H 264 or AVC Advanced Video Coding is a digital video codec standard which is noted for achieving very high data compression 2 It was developed by the ITU T Video Coding Experts Group VCEG together with the ISO IEC Moving Picture Experts Group MPEG as the product of a collective partnership effort known as the Joint Video Team JVT delivers stunning quality at remarkably low data rates 20 2 H 264 Ratified as part of the MPEG 4 standard MPEG 4 Part 10 this ultra efficient technology gives excellent results across a broad range of bandwidths from 3G 3rd Generation for mobile devices to iChat Instant chatting application for video conferencing to HD for broadcast and DVD 20 Massive Quality Minimal Files H 264 uses the latest innovations in video compression technology to provide incredible video quality from the smallest amount of video data This means we can watch crisp clear video in much smaller files saving bandwidth and storage costs over previous generations of video codecs 20 H 264 delivers the same quality as MPEG 2 at a third to half the data rate and up to four times the frame size of MPEG 4 Part 2 at the same data rate 20 Scalable from 3G to HD and Beyond H 264 achieves the best ever compression efficiency for a broad range of applications such as broadcast DVD video conferencing video on demand streaming and multimedia messaging And true to its advanced design H 264 delivers excellent quality across a wide operating range from 3G to HD and everything in between Whether the need is high quality video for a mobile 8 phone iChat Internet broadcast or satellite delivery H 264 provides exceptional performance at impressively low data rates Table 1 shows data rates used at various resolutions Use Scenario Resolution Frame Rate Example Data Rates Mobile Content 176x144 10 15 fps 50 60 Kbps Internet Standard Definition 640x480 24 fps 1 2 Mbps High Definition 1280x720 24p 5 6 Mbps Full High Definition 1920x1080 24p 7 8 Mbps Table 1 H 264 data rates at various resolutions 20 The New Industry Standard Already ratified as part of the MPEG 4 standard MPEG 4 Part 10 and the ITU T s latest video conferencing standard H 264 is now mandatory for the Blu ray specification the format for high definition DVDs and ratified in the latest versions of the DVB Digital Video Broadcasters and 3GPP 3rd Generation Partnership Project standards 20 Numerous broadcast cable videoconferencing and consumer electronics companies consider H 264 the video codec of choice for their new products and services The AVC H 264 profiles Figure 1 The H 264 AVC standard includes the following sets of capabilities which are referred to as profiles They target specific classes of applications 2 Constrained Baseline Profile CBP Primarily for low cost applications this profile is 9 used widely in videoconferencing
View Full Document
Unlocking...