University of Texas at AustinDepartment of Electrical and Computer EngineeringEE 381K: Multidimensional Digital Signal ProcessingObjective Image/Video Quality Measurement A Literature SurveyZhou WangAbstractWith the advent of various video compression standards and a proliferation ofdigital video coding products that are just beginning to appear in the marketplace, it hasbecome increasingly important to devise image/video quality assessment algorithms thatwill standardize the assessment of compressed digital image/video quality. The subjectiveassessment of Mean Opinion Score (MOS) is very tedious, expensive and cannot beconducted in real time. It is also very difficult to be embedded into a practical videoprocessing system because it cannot be implemented automatedly. In the last twodecades, there have been a lot of attempts to provide objective measures for image/videoquality. A recent trend is to incorporating Human Vision System (HVS) features into thequality metrics to make the new measurements more consistent with human visualperception. This literature survey is to give a general description on the variousconsiderations on the development and implementation of image/video qualityassessment systems.Zhou Wang, Objective Image/Video Quality Measurement – A Literature Survey Oct. 19, 1998Literature survey for EE381K: Multidimensional Digital Signal Processing course project 1I. INTRODUCTIONWith the advent of various video compression standards and a proliferation ofdigital video coding products that are just beginning to appear in the marketplace, it hasbecome increasingly important for the telecommunication, computer and mediacommunities to devise image/video quality assessment algorithms that will standardizethe objective assessment of compressed digital video quality to be utilized in multimedia,CDs, DVDs, HDTV, web-based video services, digital telephony, etc. The subjectivemeasurement Mean Opinion Score (MOS) is a widely used method on the assessment ofimage/video quality, but it has two obvious disadvantages. First, it is very tedious andexpensive, thus cannot be conducted in real time. Second, it is very difficult to beembedded into a practical video processing system because it is impossible to beimplemented automatedly. Instead, an objective image/video quality metric can provide aquality value for a given image/video automatedly in a relatively short period of time.This is very important for real world applications.In the last two decades, a lot of objective metrics have been proposed [2, 4-6, 9-24]to assess image/video quality. The easiest way to give a quality value is to use somesimple statistics features on the numerical errors between the distorted image and areference image. The most widely adopted statistics feature is the Mean Squared Error(MSE). However, MSE and its variants do not correlate well with subjective qualitymeasures because human perception of image/video distortions and artifacts isunaccounted for. MSE is also not good because the residual image is not uncorrelatedadditive noise. It contains components of the original image. A detailed discussion onMSE is given by Griod [1].A major emphasis in recent research has been given to a deeper analysis of theHuman Visual System (HVS) [2] features. There are a lot of HVS characteristics [3] thatZhou Wang, Objective Image/Video Quality Measurement – A Literature Survey Oct. 19, 1998Literature survey for EE381K: Multidimensional Digital Signal Processing course project 2may influence the human visual perception on image/video quality. Although HVS is toocomplex to fully understand with present psychophysical means, the incorporation ofeven a simplified model into objective measures reportedly leads to a better correlationwith the response of the human observers [2]. Many algorithms have successfullyemployed HVS models [2, 4, 5, 6, 10-24].Another important factor for the development of an image/video quality metric isthe flexibility for practical implementations. Some of the metrics consider only somespecial types of distortions [4] or special image/video coding methods [5, 6]. As for colorimages, finding a good color space where each color channels can be consideredindependently is desired [3, 7, 8]. The implementation for a practical video qualityassessment metric is difficult because of the computational complexity. In this case,speed is one of the major considerations.II. SIMPLE STATISTICS ERROR METRICSIn [2], a number of simple statistics metrics on numerical errors are compared forgray scale image compression. These metrics include average difference, maximumdifference, absolute error, MSE, peak MSE, Laplacian MSE, histogram, Hosaka plot (Agraphic quality measure. The area and shape of the plot gives information about the typeand amount of degradation.), etc. It is shown that although some numerical measurescorrelate well with the observers’ response for a given compression technique, they arenot reliable for an evaluation across different techniques.The major advantage of the simple statistics error metrics is their simplicity. Theycan be very conveniently adapted by an image/video processing system. However, thelacking of considering HVS features make them not good for perceptual image/videodistortion. It is shown in [2] that small improvement can be obtained by combing only avery simple HVS model.Zhou Wang, Objective Image/Video Quality Measurement – A Literature Survey Oct. 19, 1998Literature survey for EE381K: Multidimensional Digital Signal Processing course project 3III. HVS FEATURE BASED ALGORITHMSA. HVS FeaturesVarious HVS features are correlated with perceptual image/video quality [3].Among them, the most commonly used are luminance contrast sensitivity, frequencycontrast sensitivity and masking effects.The human eye is sensitive to luminance rather than the absolute luminance value.According to Weber’s law [9], if the luminance of a test stimulus is just noticeable fromthe surrounding luminance, then the ratio of just noticeable luminance difference tostimulus’ luminance, known as Weber fraction is approximately constant [10]. Inpractice, due to the present of ambient illumination surrounding the display, the noise invery dark areas tends to be less visible than that occurring in regions of higher luminance.Therefore, as the background luminance is low, the Weber fraction increases as thebackground luminance decreases [11]. On the other hand, if the background luminance