Joint Video Team JVT of ISO IEC MPEG ITU T VCEG ISO IEC JTC1 SC29 WG11 and ITU T SG16 Q 6 28th Meeting Hannover DE 20 25 July 2008 Document JVT AB031 Filename Title New Video Quality Metrics in the H 264 reference software Status Input Document to JVT Purpose Information Woo Shik Kim Peshala Pahalawatta Author s or Contact s Zhen Li Alexis Michael Tourapis Tel 1 818 823 2832 Email wkim dolby com Tel 1 818 823 2894 Email ppaha dolby com Tel 1 818 823 2845 Email zli dolby com Tel 1 818 823 2842 Email atour dolby com 3601 West Alameda Avenue Burbank CA 91505 USA Source Dolby Laboratories Inc Abstract The H 264 AVC JM reference software currently only provides encoder distortion information using Mean Square Error MSE based error metrics The computation is also constrained by the color format of the source sequence Unfortunately such metrics do not model the human visual system very accurately In this contribution the reference software was extended to support additional distortion metrics in the encoder such as the Structural Similarity Index SSIM the Multi scale SSIM MS SSIM and distortion computation in different color spaces such as RGB The introduced code is very flexible and could be further extended in the future with additional distortion metrics and could also consider different color spaces if so desired Introduction Although considerable advancements in video quality evaluation have been made in recent years video coding algorithm evaluation for standardization purposes is still based on the mean square error MSE and peak signal to noise PSNR metrics 1 5 Unfortunately it is well known that such metrics do not correlate very well with perceptual distortion 6 12 This may potentially affect the evaluation of different coding tools or algorithms during video standardization activities During the last meeting it was discussed that providing additional distortion evaluation tools within the JM reference software 13 would be highly desirable and it may help making more educated decisions on the performance or behavior of a given algorithm File Page 1 Date Saved 2008 07 20 New Distortion Metrics and the JM For this purpose we have extended the current JM reference software latest official release ver 14 0 to provide support for additional well known distortion metrics such as the Structural Similarity Index SSIM 10 and the Multi Scale Structural Similarity Index MS SSIM 12 These metrics given the assumption that the human visual system is highly affected by the structural information within a scene try to measure the structural similarity between two images therefore providing an image quality assessment score that could be used in place or be complementary to metrics such as PSNR The SSIM metric is based on the evaluation of three different measures the luminance contrast and structure comparison measures which are computed as l x y 2 x y C1 x2 y2 C1 2 C c x y 2 x y 2 2 x y C2 C3 s x y xy x y C3 where x and y correspond to two different signals that we would like to match i e two 2 different blocks in two separate images x x and xy the mean of x the variance of x and the covariance of x and y respectively while C1 C2 and C3 are constants given by C1 K1 L C2 K 2 L and C3 C2 2 L is the dynamic range for the sample data i e 2 2 L 255 for 8 bit content and K1 1 and K2 1 are two scalar constants Given the above measures the structural similarity can be computed as SSIM x y l x y c x y s x y where and define the different importance given to each measure The MS SSIM metric on the other hand is an extension of the SSIM which computes these measures at various scales and combines them using an equation of the form MSSSIM x y lM x y M M c j x y j 1 s x y j j j where M corresponds to the maximum scale we have used for our computation while j 1 corresponds the original resolution of the image In general it is considered that these metrics perform as well as or better compared to PSNR and are used in several applications for video quality evaluation purposes It should be noted that our implementation was extended to provide distortion computation assuming that the content is in YUV space also in RGB space based on the ITU R BT 601 specification The software could be easily extended to provide additional color transforms if so desired In general our implementation is very modular and extensible allowing for future distortion metrics to be introduced in the software if so desired We hope that this implementation and these tools would be useful in the creation of new video coding tools and standards by the JVT and that some of these metrics would be adopted for any future evaluations of video coding tools and proposals It may be further desirable in the future to also extend these tools in the decoder or even include them in the mode or picture coding decision mechanisms of the software Due to time and resource limitations we have not done this at this point File Page 2 Date Saved 2008 07 20 References 1 TK Tan G Sullivan T Wedi Recommended Simulation Conditions for Coding Efficiency Experiments ITU T SC16 Q6 27th VCEG Meeting Nice France 17th 18th October 2005 Doc VCEG AA10r1 2 TK Tan G Sullivan T Wedi Recommended Simulation Conditions for Coding Efficiency Experiments ITU T SC16 Q6 31st VCEG Meeting Marrakech Morocco 13th 19th January 2007 Doc VCEG AE10r1 3 TK Tan G Sullivan T Wedi Recommended Simulation Conditions for Coding Efficiency Experiments ITU T SC16 Q6 34th VCEG Meeting Antalya Turkey 12th 13th January 2008 Doc VCEG AH10r3 4 Gary Sullivan Common Conditions for Coding Efficiency Tests ITU T Q 15 16 Document VCEG N81 October 2001 This is a forthcoming revision of ITU T Q 15 16 Document VCEG M75 by Gisle Bjontegaard Recommended Simulation Conditions for H 26L 24 27 Sep 2001 Santa Barbara USA 5 Gisle Bjontegaard Calculation of Average PSNR Differences between RD curves ITU T SC16 Q6 13th VCEG Meeting Austin Texas USA April 2001 Doc VCEG M33 6 M Eskicioglu and P S Fisher Image quality measures and their performance in IEEE Transactions on Communications vol 43 pp 2959 2965 Dec 1995 7 T N Pappas and R J Safranek Perceptual criteria for image quality evaluation in Handbook of Image and Video Processing A Bovik ed Academic Press 2000 8 Z Wang and A C Bovik A universal image quality index in IEEE Signal Processing Letters vol 9 pp 81 84 Mar 2002 9 Z Wang H R Sheikh and A C Bovik Objective video quality assessment in The Handbook of Video Databases Design and Applications B Furht and O
View Full Document
Unlocking...