Babu Hemanth Kumar [email protected]. K.R.Rao In the rate-distortion optimization for H.264 I-frame encoder, the distortion (D) is measured as the sum of the squared differences between the reconstructed and the original blocks, which is MSE. Although PSNR and MSE are currently the most widely used objective metrics due to their low complexity and clear physical meaning, they were also widely criticized for not correlating well with Human Visual System (HVS) [2] for a long time. The study from previous literature shows that structural similarity metric provides better image assessment than pixel error based metric (mean square error and peak signal-to-noise ratio).2IntroductionMean Squared Error: Love It or Leave It?3 So what is the secret of the MSE—why is it still so popular? What is wrong with the MSE when it does not work well? Just how wrong is the MSE in these cases? If not the MSE, what else can be used?What is MSE?4 MSE is a signal fidelity measure.The goal of a signal fidelity measure is to compare two signals byproviding a quantitative score that describes the degree of similarity/fidelity or, conversely, the level of error/distortion between them. Suppose that x = {xi |i = 1, 2, · · · , N} and y = {yi |i =1, 2, · · · , N} are two finite-length, discrete signals , where N is the number ofsignal samples and xi and yi are the values of the i th samples in x and y, respectively. The MSE between the signals isThe MSE has many attractive features: It is simple. It is parameter free and inexpensive to compute, with a complexity of only one multiply and two additions per sample. It is also memoryless—the squared error can be evaluated at each sample, independent of other samples. It has a clear physical meaning—it is the natural way to define the energy of the error signal The MSE is an excellent metric in the context of optimization MSE is widely used simply because it is a convention5Why do we love MSE?[FIG1] Comparison of image fidelity measures for “Einstein” image altered with different types of distortions. (a)Reference image. (b) Mean contrast stretch. (c) Luminance shift. (d) Gaussian noise contamination. (e) Impulsive noise contamination. (f) JPEGcompression. (g) Blurring. (h) Spatial scaling (zooming out). (i) Spatial shift (to the right). (j) Spatial shift (to the left). (k) Rotation(counter-clockwise). (l) Rotation (clockwise).[2]6What is wrong with MSE?7Implicit Assumptions when using MSE Signal fidelity is independent of temporal or spatial relationships between the samples of the original signal. If the original and distorted signals are randomly re-ordered in the same way, then the MSE between them will be unchanged. Signal fidelity is independent of any relationship between the original signal and the error signal. For a given error signal, the MSE remains unchanged, regardless of which original signal it is added to. Signal fidelity is independent of the signs of the error signal samples.All signal samples are equally important to signal fidelity.8Failures of MSE Metric[FIG2] Failures of MSE Metric [2]Alternative Approach[FIG3] Examples of structural versus nonstructural distortions.[2]9If we view the HVS as an ideal information extractor that seeks to identify and recognize objects in the visual scene, then it must be highly sensitive to the structural distortions and automatically compensates for the nonstructural distortions. Consequently, an effective objective signal fidelity measure should simulate this functionality Recent proposed approach for image quality assessment Method for measuring the similarity between two images. Full reference metrics The SSIM is designed to improve on traditional metricslike PSNR and MSE, which have proved to beinconsistent with human eye perception.10SSIMProperty of SSIM Value lies between [0,1] Symmetry: S(x,y) = S(y,x) Boundedness: S(x,y) <= 1 Unique maximum: S(x,y) = 1 if and only if x=y (in discrete representations xi= yi, for all i= 1,2…….,N ).11SSIM Measurement System12[FIG4] Block Diagram of Structural Similarity measurement system[4]13H.264 [FIG 5] Block Diagram of H.264 encoder[FIG 6]. Intra 4 x 4 prediction mode directions (vertical : 0, horizontal : 1, DC : 2, diagonal down left : 3, diagonal down right : 4, vertical right : 5, horizontal down : 6, vertical left : 7, horizontal up : 8)[5]14Intra-prediction H.264 is able to gain much of its efficiency by simplifying redundant data not only across a series of frames, but also within a single frame, a technique called intraframe prediction [FIG 6]. The H.264 encoder uses intraframe prediction with more ways to reference neighboring pixels, so it compresses details and gradients better than previous codecs.H.264 I-Frame Encoder 15 The best prediction mode(s) are chosen utilizing the R-D optimization which is described as:J (s ,c,MODE | QP) = D(s , c,MODE | QP) + MODE * R(s,c ,MODE | QP) Distortion D(s,c,MODE|QP) is measured as SSD between the original block s and the reconstructed block c, and QP is the quantization parameter, MODE is the prediction mode. R(s,c,MODE|QP) is the number of bits coding the block. The modes(s) with the minimum J(s,c,MODE|QP) are chosen as the prediction mode(s) of the macroblock.Proposal16 The main idea of this project is to employ SSIM in the rate-distortion optimizations of H.264 I-frame encoder to choose the best prediction mode(s). The required modifications will be done on the JVT reference software JM92 program. Results in terms of total number of bits of the compressed image, SSIM of the whole reconstructed image for H.264-JM92 software and the new method will be compared. The quality of the reconstructed picture is higher when its SSIM index is greater while the SSD performs the other way. Therefore the distortion in this method is measured as: D (s, c, MODE|QP)== 1−SSIM(s, c) s and c are the original and reconstructed image block resp.The new-Rate Distortion can now be written as :J (s , c,MODE|QP) = 1 - SSIM(s , c) + MODE * R( s, c,MODE |QP) The algorithm uses SSIM index instead of SSD as the distortion measure in RDCost_for_4x4IntraBlock, RDCost_for_8x8IntraBlock and RDCost_for_macroblocks of H.264-JM92 software.Proposal Method18Test SequencesCoastguard Akiyo Bridge-close Car phoneClaire Container Grandma Miss-America19Simulation
View Full Document
Unlocking...