AbstractIntroductionNew Distortion Metrics and the JMReferences(Append for Proposal Documents)JVT Patent Disclosure FormJoint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG(ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6)28th Meeting: Hannover, DE, 20-25 July, 2008Document: JVT-AB031Filename: Title: New Video Quality Metrics in the H.264 reference softwareStatus: Input Document to JVTPurpose: InformationAuthor(s) orContact(s):Woo-Shik Kim Tel: +1-818-823-2832Email: [email protected] Peshala Pahalawatta Tel: +1-818-823-2894Email: [email protected] Zhen Li Tel: +1-818-823-2845Email: [email protected] Alexis Michael Tourapis Tel: +1-818-823-2842Email: [email protected] 3601 West Alameda AvenueBurbank, CA 91505USASource: Dolby Laboratories Inc._____________________________AbstractThe H.264/AVC JM reference software currently only provides encoder distortion informationusing Mean Square Error (MSE) based error metrics. The computation is also constrained bythe color format of the source sequence. Unfortunately, such metrics do not model the humanvisual system very accurately. In this contribution the reference software was extended tosupport additional distortion metrics in the encoder such as the Structural Similarity Index(SSIM), the Multi-scale SSIM (MS-SSIM), and distortion computation in different color spacessuch as RGB. The introduced code is very flexible and could be further extended in the futurewith additional distortion metrics and could also consider different color spaces if so desired.IntroductionAlthough considerable advancements in video quality evaluation have been made in recentyears, video coding algorithm evaluation for standardization purposes is still based on the meansquare error (MSE) and peak signal to noise (PSNR) metrics [1]-[5]. Unfortunately, it is wellknown that such metrics do not correlate very well with perceptual distortion [6]-[12]. This maypotentially affect the evaluation of different coding tools or algorithms during videostandardization activities. During the last meeting it was discussed that providing additionaldistortion evaluation tools within the JM reference software [13] would be highly desirable and itmay help making more educated decisions on the performance or behavior of a given algorithm.File: Page: 1 Date Saved: 2008-07-20New Distortion Metrics and the JMFor this purpose, we have extended the current JM reference software (latest official releasever. 14.0) to provide support for additional well known distortion metrics such as the StructuralSimilarity Index (SSIM) [10] and the Multi-Scale Structural Similarity Index (MS-SSIM) [12].These metrics given the assumption that the human visual system is highly affected by thestructural information within a scene, try to measure the structural similarity between twoimages, therefore providing an image quality assessment score that could be used in place orbe complementary to metrics such as PSNR. The SSIM metric is based on the evaluation ofthree different measures, the luminance, contrast, and structure comparison measures whichare computed as:,2),(1221CClyxyxyx,2),(2222CCcyxyxyx,),(33CCsyxxyyxwhere xand y correspond to two different signals that we would like to match, i.e. twodifferent blocks in two separate images,x, 2x, and xythe mean of x, the variance of x,and the covariance of xand yrespectively, while C1, C2, and C3 are constants given by 211LKC , 222LKC , and 2/23CC . L is the dynamic range for the sample data, i.e.L=255 for 8 bit content and K1<<1 and K2<<1 are two scalar constants. Given the abovemeasures the structural similarity can be computed as ),(),(),(),( yxyxyxyx sclSSIM where , , and define the different importance given to each measure. The MS-SSIMmetric, on the other hand, is an extension of the SSIM which computes these measures atvarious scales and combines them using an equation of the form: MjjjMjjMsclMSSSIM1),(),(),(),(yxyxyxyxwhere M corresponds to the maximum scale we have used for our computation, while j=1corresponds the original resolution of the image. In general, it is considered that these metricsperform as well as or better compared to PSNR and are used in several applications for videoquality evaluation purposes.It should be noted that our implementation was extended to provide distortion computation,assuming that the content is in YUV space, also in RGB space based on the ITU-R BT.601specification. The software could be easily extended to provide additional color transforms if sodesired. In general, our implementation is very modular and extensible, allowing for future distortionmetrics to be introduced in the software, if so desired. We hope that this implementation andthese tools would be useful in the creation of new video coding tools and standards by the JVT,and that some of these metrics would be adopted for any future evaluations of video codingtools and proposals. It may be further desirable in the future to also extend these tools in thedecoder, or even include them in the mode or picture coding decision mechanisms of thesoftware. Due to time and resource limitations we have not done this at this point.File: Page: 2 Date Saved: 2008-07-20References[1] TK Tan, G. Sullivan, T. Wedi, “Recommended Simulation Conditions for Coding EfficiencyExperiments”, ITU-T SC16/Q6, 27th VCEG Meeting, Nice, France, 17th - 18th October2005, Doc.VCEG-AA10r1.[2] TK Tan, G. Sullivan, T. Wedi, “Recommended Simulation Conditions for Coding EfficiencyExperiments”, ITU-T SC16/Q6, 31st VCEG Meeting, Marrakech, Morocco, 13th – 19thJanuary, 2007, Doc.VCEG-AE10r1.[3] TK Tan, G. Sullivan, T. Wedi, “Recommended Simulation Conditions for Coding EfficiencyExperiments”, ITU-T SC16/Q6, 34th VCEG Meeting, Antalya, Turkey, 12th – 13th January,2008, Doc.VCEG-AH10r3.[4] Gary Sullivan, “Common Conditions for Coding Efficiency Tests”, ITU-T Q.15/16, DocumentVCEG-N81, October 2001. This is a forthcoming revision of ITU-T Q.15/16, DocumentVCEG-M75 by Gisle Bjontegaard, “Recommended Simulation Conditions for H.26L”, 24-27Sep 2001, Santa
View Full Document