Babu Hemanth Kumar Aswathappa babuhemanthkumar aswathappa mavs uta edu Guidance Dr K R Rao Introduction In the rate distortion optimization for H 264 I frame encoder the distortion D is measured as the sum of the squared differences between the reconstructed and the original blocks which is MSE Although PSNR and MSE are currently the most widely used objective metrics due to their low complexity and clear physical meaning they were also widely criticized for not correlating well with Human Visual System HVS 2 for a long time The study from previous literature shows that structural similarity metric provides better image assessment than pixel error based metric mean square error and peak signal to noise ratio 2 Mean Squared Error Love It or Leave It So what is the secret of the MSE why is it still so popular What is wrong with the MSE when it does not work well Just how wrong is the MSE in these cases If not the MSE what else can be used 3 What is MSE MSE is a signal fidelity measure The goal of a signal fidelity measure is to compare two signals by providing a quantitative score that describes the degree of similarity fidelity or conversely the level of error distortion between them Suppose that x xi i 1 2 N and y yi i 1 2 N are two finite length discrete signals where N is the number of signal samples and xi and yi are the values of the i th samples in x and y respectively The MSE between the signals is 4 Why do we love MSE The MSE has many attractive features It is simple It is parameter free and inexpensive to compute with a complexity of only one multiply and two additions per sample It is also memoryless the squared error can be evaluated at each sample independent of other samples It has a clear physical meaning it is the natural way to define the energy of the error signal The MSE is an excellent metric in the context of optimization MSE is widely used simply because it is a convention 5 What is wrong with MSE FIG1 Comparison of image fidelity measures for Einstein image altered with different types of distortions a Reference image b Mean contrast stretch c Luminance shift d Gaussian noise contamination e Impulsive noise contamination f JPEGcompression g Blurring h Spatial scaling zooming out i Spatial shift to the right j Spatial shift to the left k Rotation counter clockwise l Rotation clockwise 2 6 Implicit Assumptions when using MSE Signal fidelity is independent of temporal or spatial relationships between the samples of the original signal If the original and distorted signals are randomly re ordered in the same way then the MSE between them will be unchanged Signal fidelity is independent of any relationship between the original signal and the error signal For a given error signal the MSE remains unchanged regardless of which original signal it is added to Signal fidelity is independent of the signs of the error signal samples All signal samples are equally important to signal fidelity 7 Failures of MSE Metric FIG2 Failures of MSE Metric 2 8 Alternative Approach If we view the HVS as an ideal information extractor that seeks to identify and recognize objects in the visual scene then it must be highly sensitive to the structural distortions and automatically compensates for the nonstructural distortions Consequently an effective objective signal fidelity measure should simulate this functionality FIG3 Examples of structural versus nonstructural distortions 2 9 SSIM Recent proposed approach for image quality assessment Method for measuring the similarity between two images Full reference metrics The SSIM is designed to improve on traditional metrics like PSNR and MSE which have proved to be inconsistent with human eye perception 10 Property of SSIM Value lies between 0 1 Symmetry S x y S y x Boundedness S x y 1 Unique maximum S x y 1 if and only if x y in discrete representations xi yi for all i 1 2 N 11 SSIM Measurement System FIG4 Block Diagram of Structural Similarity measurement system 4 12 H 264 FIG 5 Block Diagram of H 264 encoder 13 Intra prediction H 264 is able to gain much of its efficiency by simplifying redundant data not only across a series of frames but also within a single frame a technique called intraframe prediction FIG 6 The H 264 encoder uses intraframe prediction with more ways to reference neighboring pixels so it compresses details and gradients better than previous codecs FIG 6 Intra 4 x 4 prediction mode directions vertical 0 horizontal 1 DC 2 diagonal down left 3 diagonal down right 4 vertical right 5 horizontal down 6 vertical left 7 horizontal up 8 5 14 H 264 I Frame Encoder The best prediction mode s are chosen utilizing the R D optimization which is described as J s c MODE QP D s c MODE QP MODE R s c MODE QP Distortion D s c MODE QP is measured as SSD between the original block s and the reconstructed block c and QP is the quantization parameter MODE is the prediction mode R s c MODE QP is the number of bits coding the block The modes s with the minimum J s c MODE QP are chosen as the prediction mode s of the macroblock 15 Proposal The main idea of this project is to employ SSIM in the rate distortion optimizations of H 264 I frame encoder to choose the best prediction mode s The required modifications will be done on the JVT reference software JM92 program Results in terms of total number of bits of the compressed image SSIM of the whole reconstructed image for H 264 JM92 software and the new method will be compared 16 Proposal Method The quality of the reconstructed picture is higher when its SSIM index is greater while the SSD performs the other way Therefore the distortion in this method is measured as D s c MODE QP 1 SSIM s c s and c are the original and reconstructed image block resp The new Rate Distortion can now be written as J s c MODE QP 1 SSIM s c MODE R s c MODE QP The algorithm uses SSIM index instead of SSD as the distortion measure in RDCost for 4x4IntraBlock RDCost for 8x8IntraBlock and RDCost for macroblocks of H 264 JM92 software Test Sequences Coastguard Claire Akiyo Container Bridge close Grandma Car phone Miss America 18 Simulation Results TABLE 1 Results of comparison between H 264 JM92 and H 264 JM92 SSIM method for QP 30 19 Simulation Results TABLE 2 Results of comparison between H 264 JM92 and H 264 JM92 SSIM method for QP 20 20 Simulation Results TABLE 3 Results of comparison between H 264 JM92 and H 264 JM92 SSIM method for QP 10 21 Simulation Results Coastguard Original Encoded by H 264 encoder with QP 30
View Full Document
Unlocking...