Vol 447 10 May 2007 doi 10 1038 nature05724 LETTERS Image statistics and the perception of surface qualities Isamu Motoyoshi1 Shin ya Nishida1 Lavanya Sharan2 Edward H Adelson2 The world is full of surfaces and by looking at them we can judge their material qualities Properties such as colour or glossiness can help us decide whether a pancake is cooked or a patch of pavement is icy Most studies of surface appearance have emphasized textureless matte surfaces1 3 but real world surfaces which may have gloss and complex mesostructure are now receiving increased attention4 7 Their appearance results from a complex interplay of illumination reflectance and surface geometry which are difficult to tease apart given an image If there were simple image statistics that were diagnostic of surface properties it would be sensible to use them8 11 Here we show that the skewness of the luminance histogram and the skewness of sub band filter outputs are correlated with surface gloss and inversely correlated with surface albedo diffuse reflectance We find evidence that human observers use skewness or a similar measure of histogram asymmetry in making judgements about surfaces When the image of a surface has positively skewed statistics it tends to appear darker and glossier than a similar surface with lower skewness and this is true whether the skewness is inherent to the original image or is introduced by digital manipulation We also find a visual aftereffect based on skewness adaptation to patterns with skewed statistics can alter the apparent lightness and glossiness of surfaces that are subsequently viewed We suggest that there are neural mechanisms sensitive to skewed statistics and that their outputs can be used in estimating surface properties Figure 1 shows two renderings of a three dimensional model of Michelangelo s sculpture of St Matthew12 The version on the left appears darker and glossier than the one on the right This is true even though the two images have been scaled to have the same mean luminance We are unaware of any theories that will predict the changes in lightness or gloss that we observe The image of a surface arises from the combination of the surface geometry the surrounding illumination and the surface optics Each of these components can be complex for example the reflectance at each point is characterized by a four dimensional function known as the bidirectional reflectance distribution function13 Each is typically unknown and estimating any one using inverse optics requires knowing the others To bypass this problem we have looked for simple statistical image measurements that can provide information that is useful even if not complete Any two dimensional image measurements that are statistically related to properties of the threedimensional scene are potentially useful8 11 We made a set of patches of stucco like material The values of albedo and glossiness were uniform within each patch but they were varied systematically from one patch to another by changing paint pigmentation and acrylic media coating respectively We photographed these objects linearized the pixel values and normalized the mean luminance by multiplicative scaling We found that changes in albedo and glossiness were accompanied by characteristic changes in the luminance histogram Consider the two stucco patches of Figure 1 These two synthetic images of Michelangelo s St Matthew sculpture have the same mean luminance The one on the left looks darker and glossier than the one on the right 1 Human and Information Science Lab NTT Communication Science Labs Nippon Telegraph and Telephone Corporation 3 1 Morinosato Wakamiya Atsugi 243 0198 Japan Department of Brain and Cognitive Sciences and Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology 43 Vassar Street 46 4115 Cambridge Massachusetts 02139 USA 2 206 2007 Nature Publishing Group LETTERS NATURE Vol 447 10 May 2007 Number of pixels a 0 1 0 3 Skewness 2 0 1 4 Lightness rating 0 c 3 Glossiness rating Lightness rating 3 2 1 2 0 2 Skewness 0 1 Skewness 2 1 10 100 Specular intensity 3 2 1 2 0 2 Skewness 4 4 Glossiness rating Lightness rating 3 Glossiness rating 2 0 4 4 4 3 2 1 0 4 1 4 4 d 3 0 0 1 2 1 10 100 Diffuse reflectance 0 4 0 5 Luminance Skewness Lightness rating 4 Glossiness rating b 0 5 Luminance Skewness Fig 2a In comparison with a light matte surface left a dark glossy surface right has a long positive tail In general as the albedo of glossy surfaces is decreased or as the glossiness is increased regardless of the albedo the histogram s skewness tends to increase Fig 2b black circles These changes make sense given the influence of specular and diffuse reflectance on the appearance of specular highlights Highlights are stronger and sharper on glossy surfaces and they have higher contrast when viewed on darker surfaces because they are seen against a body surface that has a lower luminance Having observed this physical relationship we next looked for a corresponding psychophysical relationship We showed these stucco images one by one to human observers presenting them against a dark background on a monitor at constant mean luminance and asked the observers to rate the lightness perceived diffuse reflectance or glossiness of each surface The judgments were well correlated with the corresponding physical properties as shown in Fig 2b red circles Both the lightness and glossiness ratings were also well correlated with the skewness of the luminance histogram Fig 2c to a degree comparable with or even higher than the correlations with corresponding physical properties r 5 20 87 for correlation with skewness of lightness ratings and 0 89 for glossiness ratings respectively We next chose a set of images of three materials stucco black cotton fabric and crumpled white paper all of which were surfaces of uniform albedo and glossiness and used a lookup table to force the luminance histograms to have specific skewness values As expected the lightness rating showed a strong negative dependency on skewness whereas the glossiness rating showed a strong positive dependency This was true for each image class Fig 2d Further tests of a wide variety of materials gave similar results described in Supplementary Data A In addition to the effects of skewness we found a minor effect of the standard deviation of the luminance histogram on both lightness and glossiness The mean luminance had a significant effect on lightness1 3 but
View Full Document
Unlocking...