Geisler & Perry 6/20/2008 1Contour Statistics in Natural Images: Grouping Across Occlusions Wilson S. Geisler and Jeffrey S. Perry Center for Perceptual Systems and Department of Psychology University of Texas at Austin Abstract Correctly interpreting a natural image requires dealing properly with the effects of occlusion, and hence contour grouping across occlusions is a major component of many natural visual tasks. To better understand the mechanisms of contour grouping across occlusions we (a) measured the pair-wise statistics of edge elements from contours in natural images, as a function of edge-element geometry and contrast polarity, (b) derived the ideal Bayesian observer for a contour occlusion task where the stimuli were extracted directly from natural images, and then (c) measured human performance in the same contour occlusion task. In addition to discovering new statistical properties of natural contours, we found that naïve human observers closely parallel ideal performance in our contour occlusion task. In fact, there was no region of the four-dimensional stimulus space (3 geometry dimensions and 1 contrast dimension) where humans did not closely parallel the performance of the ideal observer (i.e., efficiency was approximately constant over the entire space). These results reject many other contour grouping hypotheses and strongly suggest that the neural mechanisms of contour grouping are tightly related to the statistical properties of contours in natural images. Introduction It is common in natural scenes for an object to be partially occluded by one or more other objects (Fig. 1). Such occlusions can provide useful depth and segmentation (figure-ground) information; for example, if the bounding contour of an object can be identified, then other contours intersecting that bounding contour are likely to be occluded, and hence likely to be at a greater distance and to derive from a different physical source than the bounding contour (e.g., a different object). However, the existence of occlusions can also greatly increase the difficulty of correctly interpreting natural images; for example, an occluding object necessarily obscures image features from the occluded objects, making identification of the occluded objects difficult.Geisler & Perry 6/20/2008 2 The human visual system contains powerful contour grouping mechanisms that are thought to play an important role in helping the visual system both exploit occlusions and overcome the loss of features produced by occlusions (e.g., Rock 1975; Barrow & Tenenbaum 1986; Kellman 2003). For example, contour grouping mechanisms allow us to decide (correctly) that the two contours passing under the red leaf in Fig. 1 arise from the same physical source (surface boundary). These contour grouping mechanisms undoubtedly evolved and/or develop in response to the properties of natural environments, and thus there have been recent efforts to directly measure the statistical properties of contours in natural images, with the aim of gaining a deeper understanding of the image information available to support contour grouping and of developing more refined models of contour grouping (Geisler et al. 2001; Elder & Goldberg 2002; Martin et al. 2004). Figure 1. Examples of occlusion in natural scenes. One approach has been to extract contour elements from natural images using an automatic edge detection algorithm and then examine the pair-wise statistics of the extracted contour elements. Using this approach Sigman et al. (2001) and Geisler et al. (2001) examined the statistics of the geometrical relationship between contour elements and found that there is a local maximum in the pair-wise probability distribution for edge elements that are approximately co-circular (i.e.,Geisler & Perry 6/20/2008 3are approximately tangent to a common circle, but see later). Geisler et al. (2001) also showed that there is a larger local maximum for edge elements that are approximately parallel (i.e., are approximately tangent to parallel lines). These two properties undoubtedly reflect the fact that natural contours are relatively smooth (e.g., the bounding contours of a branch or leaf in Fig. 1), and that natural images contain many parallel contours (e.g., the two sides of a branch or leaf in Fig. 1). Such measurements of pair-wise statistics can identify important statistical structure that is relevant for grouping; however, the measurements are obtained completely within the domain of images. Potentially more relevant statistical relationships can be obtained by measuring across-domain statistics, which involves measurements both within images as well as within the corresponding environments in order to obtain ground truth information. Measurements of ground truth are essential for determining how a rational observer should use image information when interacting with, or drawing inferences about, the environment. (For more discussion of the distinction between within-domain and across-domain statistics see Geisler 2008.) Direct measurement of ground truth information can be difficult, and thus a common shortcut is to exploit hand segmentation by human observers (Brunswik & Kamia 1953; Geisler, et al. 2001; Elder & Goldberg 2002; Martin et al. 2004; others). The premise of this approach is that under some circumstances human observers can make veridical assignments of image pixels to physical sources in the environment. To the extent that this assumption holds (see later), the assignment data can provide useful ground truth information. The present study uses the hand segmentation database for natural images reported in Geisler et al. (2001). In that study we applied an automatic algorithm to detect local contour elements (at a small spatial scale) in a diverse collection of natural images, and then had observers assign the elements to physical sources (surface/material boundary contours, shadow/shading contours, or surface marking contours). The earlier study only considered the statistics of the geometrical relationship between contour elements. In the current study, we extend the statistical analysis to include the contrast relationship between contour elements (specifically the contrast polarity). We then describe a contour grouping experiment where subjects are required to decide whether a pair of contour elements at the boundary of an occluder belongs to the same or different physicalGeisler & Perry
View Full Document
Unlocking...