Error concealment techniques in H 264 video transmission over wireless networks M U LT I M E D I A P R O C E S S I N G E E 5 3 5 9 SPRING 2011 DR K R RAO PROJECT PROPOSAL Murtaza Mustafa Zaveri murtaza zaveri mavs uta edu 1000671952 The H 264 Standard Newest entry in the series of international video coding standards Developed by a Joint Video Team JVT consisting of experts from VCEG and MPEG Design provides the most current balance between the coding efficiency implementation complexity and cost Has improved coding efficiency by a factor of at least about two on average over MPEG 2 6 The H 264 Standard contd Block diagram of a H 264 encoder 18 The H 264 Standard contd Block diagram of a H 264 decoder 18 Problem Statement Imperfections in the communication channel often result in packet loss which in turn lead to frame loss or corrupted areas in the decoded frame As H 264 employs predictive coding this kind of corruption spreads spatio temporally to the current and consecutive frames H 264 is thus susceptible to error propagation due to channel noise leading to a considerable degradation in the video quality 1 Illustration of spatio temporal error propagation 14 Error Propagation Illustration of error propagation 17 Error Resilience For better coding efficiency the H 264 standard gives strong emphasis to error resiliency and the adaptability to various networks H 264 AVC has adopted a two layer structure design containing a video coding layer VCL which is designed to obtain highly compressed video data and a network abstraction layer NAL which formats the VCL data and adds corresponding header information for adaptation to various transportation protocols or storage media 14 Error Resilience contd VCL NAL layers of H 264 14 Error Resilient Video Coding For video coding a frame is divided into Macro Blocks MBs For each MB motion estimation finds the best match from the reference frame s by minimizing the difference between the current MB and the candidate MBs from the reference frames These residual MBs form a residual frame that is essentially the difference between the current frame and the corresponding motion compensated predicted frame Simultaneously motion vectors MVs are used to encode the locations of MBs that have been used to each MB in the current frame The residual frame is then transformed through DCT or integer transform and quantized 14 Error Resilient Video Coding contd To handle the errors the following stages are required in an error resilient decoder 14 Error detection and localization Resynchronization Error concealment Error Resilient Video Coding contd Error detection is done with the help of video syntax and or semantics When violation of video semantics syntax is observed decoder reports an error and tries to resynchronize at the next start code H 264 test model is based on the assumption that the data recovery does not bring a significant advantage to the reconstructed frames Therefore the corrupted packets are simply discarded and the lost region of video frame is concealed The error concealment schemes try to minimize the visual artifacts due to errors Error Concealment The main task of error concealment is to replace missing parts of the video content by previously decoded parts of the video sequence in order to eliminate or reduce the visual effects of errors caused by corrupted areas in the decoded frame 2 Error concealment exploits the spatial and temporal correlations between the neighboring image parts within the same frame or from the past and future frames Typical parameters used to evaluate the quality of reconstruction include peak signal to noise ratio PSNR and structural similarity index metric SSIM 18 Error Concealment Techniques Spatial domain error concealment Each missing pixel of the corrupted image part is interpolated from the intact surroundings pixels 3 Example Weighted averaging a Edge detection on boundary pixels b Detected edge points c Obtaining representative edge points d Edge matching and linking 16 Error Concealment Techniques contd Original Error Weighted Average Error Concealment Techniques contd Temporal domain error concealment replaces the missing image part with the spatially corresponding part inside a previously decoded frame which has maximum correlation with the affected frame 4 Example Motion vector estimation Compensate the missed MB by extrapolating each MV that is stored in the previously decoded frame Multi frame motion estimation 17 Error Concealment Techniques contd a Original sequence b Distorted sequence c Concealed output using motion estimation 17 Objective To implement both the spatial domain and temporal domain categories of error concealment techniques on H 264 with the application of the Joint Model JM Reference software 19 To use metrics like the peak signal to noise ratio PSNR and the structural similarity index metric SSIM 18 in order compare and evaluate the quality of reconstruction To implement a hybrid error concealment technique and attempt to demonstrate its superiority thereby concluding that its application is more advantageous over the two techniques mentioned above References 1 Y Xu and Y Zhou H 264 Video Communication Based Refined Error Concealment Schemes IEEE Transactions on Consumer Electronics vol 50 issue 4 pp 1135 1141 November 2004 2 M Wada Selective Recovery of Video Packet Loss using Error Concealment IEEE Journal on Selected Areas in Communication vol 7 issue 5 pp 807 814 June 1989 3 Y Chen et al An Error Concealment Algorithm for Entire Frame Loss in Video Transmission Microsoft Research Asia Picture Coding Symposium December 2004 4 H Ha C Yim and Y Y Kim Packet Loss Resilience using Unequal Forward Error Correction Assignment for Video Transmission over Communication Networks ACM digital library on Computer Communications vol 30 pp 3676 3689 December 2007 5 X Xiu L Zhuo and L Shen A hybrid error concealment method based on H 264 standard 8th International Conference on Signal Processing vol 2 April 2006 6 G Sullivan P Topiwala and A Luthra The H 264 AVC Advanced Video Coding Standard Overview and Introduction to the Fidelity Range Extensions SPIE Conference on Applications of Digital Image Processing XXVII vol 5 pp 454 474 November 2004 7 R Schafer T Wiegand and H Schwarz The emerging H 264 AVC standard EBU Technical Review Special Issue on Best of 2003 January 2003 8 T Wiegand et al Overview of the H 264 AVC Video Coding Standard IEEE Transactions on Circuits and Systems
View Full Document
Unlocking...