Research Topic Error Concealment Techniques in H 264 AVC for Wireless Video Transmission in Mobile Networks July 22nd 2008 Vineeth Shetty Kolkeri EE Graduate UTA 1 Outline 2 Introduction 3 Error control 4 Error concealment techniques 5 Implementation and quality assessment metric 6 Future work 2 1 Introduction Figure 1 Typical Situation of 3G 4G cellular telephony 3 Figure 1 1 Basic Coding Structure for H 264 1 4 1 1 H 264 Encoder Figure 1 2 Encoder 10 5 1 2 H 264 Decoder Figure 1 3 Decoder 10 6 Purpose of H 264 MPEG 4 part 10 1 Higher coding efficiency than previous standards MPEG 1 2 4 part 2 H 261 H 263 2 Simple syntax specifications 3 Seamless integration of video coding into all current protocols 4 More error robustness 5 Various applications like video broadcasting video streaming video conferencing D Cinema HDTV 6 Network friendliness 7 Balance between coding efficiency implementation complexity and cost based on state of the art in VLSI design technology 7 Better image quality at the same compressed bitrate or a lower compressed bitrate for the same image quality Figure 1 4 PSNR between original and reconstructed pictures and bit rate saving results of Tempete CIF 15Hz sequence for the video streaming application 10 8 2 Error Control Figure 2 1 Wireless Video Applications MMS PSS and PCS differentiation by real time or offline processing for encoding transmission and decoding 11 Figure 2 2 Packet Transmission in wireless medium Goal of Error Control Overcome the effects of errors during the transmission of the video frames in the wireless medium e g packet loss on a packet network or a wireless network Method used for Error Control Error Concealment 9 3 Error Concealment 4 Problem Transmission errors may result in lost information 2 Goal Estimate the lost information in order to conceal the fact that an error has occurred 3 Error concealment is performed at the decoder 4 Observation Video exhibits a significant amount of correlation along the spatial and temporal dimensions 5 Basic approach Perform some form of spatial temporal concealment to estimate the lost information from the correctly received data 10 Error Concealment cont Consider the case where a single macroblock 16x16 block of pixels is lost Three examples of error concealment 1 Spatial Concealment Estimate missing pixels by smoothly extrapolating surrounding pixels Correctly recovering missing pixels is extremely difficult however even correctly estimating the DC average value is very helpful 2 Temporal Concealment Copy the pixels at the same spatial location in the previous frame Effective when there is no motion potential problems when there is motion 3 Motion compensated temporal Concealment Estimate missing block as motion compensated block from previous frame Can use coded motion vector neighboring motion vector or compute new motion vector 11 3 1 Motion Vector Extrapolation MVE Compensate the missed MB by extrapolating each MV that is stored in previously decoded frame 2 8x8 sub block based process 3 Large overlapped MV is selected for the sub block If there is no overlap then use Zero MV Figure 3 1 Motion vectors from the previous frame 4 12 4 Error Concealment MB missing Zero MV Replaces missed MV as 0 0 Copy a macro block from previously reconstructed reference slice at the exact same position Figure 4 1 Zero MV concealment in dispersed FMO slices 13 4 1 Error Concealment Frame missing 1 Temporal Replacement Copy a MB Frame from previously reconstructed reference slice at the exact same position 2 Motion Vector Copy Exploits MVs of a few past frames Estimate the MV of each pixel in last successful frame Project last frame onto an estimate of missing frame 14 4 1 1 Temporal Replacement Frame Copy Figure 4 2 Frames 5 6 and 7 of the Original Sequence Figure 4 3 Frame 5 of the decoded frame Successfully decoded lost Frame 15 6 Frame 6 was reconstructed by Frame copy Frame 7 is degraded Inter temporal prediction block based motion estimation and compensation 1 Multiple reference pictures 2 Reference P pictures 3 Arbitrary referencing order 4 Variable block sizes for motion compensation Seven block sizes 16x16 16x8 8x16 8x8 8x4 4x8 4x4 5 1 4 sample luma interpolation 1 4 or 1 8th sample chroma interpolation in 4 2 0 format 6 Weighted prediction 7 Frame or field based motion estimation for interlaced scanned video 16 4 1 2 Motion Vector Copy Figure 4 4 Frames 5 6 and 7 of the Original Sequence Figure 4 5 Frame 5 of the decoded frame Successfully decoded lost Frame 6 Frame 17 6 was reconstructed by Motion Copy algorithm Frame 7 is degraded Figure 4 7 Frame divided into multiple macroblocks of 16 x 16 8 x 8 4 x 4 variable size to represent coding profiles No of bits in I and P frames Figure 4 8 Graph shows the size of the different I and P frames obtained after encoding 19 frames of the foreman QCIF video sequence Green line shows the average values of the bit lost when it is passed through the lossy algorithm after encoding in a video 18 sequence 0 97 0 96 0 95 0 94 Series2 0 93 Series1 0 92 0 91 0 9 0 89 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Figure 4 9 Comparison of the recovered frames using Frame copy using SSIM 0 97 0 96 0 95 0 94 Series2 Series1 0 93 0 92 0 91 0 9 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Figure 4 10 Comparison on of the recovered frames using Motion Estimation 19 using SSIM 38 37 36 35 34 Series2 33 Series1 32 31 30 29 28 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Figure 4 11 Comparison of the recovered frames using Frame copy using PSNR 40 35 30 25 Series2 20 Series1 15 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Figure 4 12 Comparison of the recovered frames using Motion Estimation 20 using PSNR 60 50 40 Series2 30 Series1 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Figure 4 13 Comparison of the recovered frames using Frame copy using MSE 60 50 40 Series3 30 Series2 Series1 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Figure 4 14 Comparison of the recovered frames using Frame copy using 21 MSE 5 Different Error Concealment Techniques Ref I C Todoli Performance of Error Concealment Methods for Wireless Video Diploma Thesis Vienna University of Technology 2007 5 Original Error Weighted Average Decode I Frame without residuals Decode without residuals Copy paste Boundary matching Block matching 22 Weighted averaging The often used method averaging Each pixel of a missing block …
View Full Document
Unlocking...