Vision Research 41 (2001) 711–724Edge co-occurrence in natural images predicts contour groupingperformanceW.S. Geislera,*, J.S. Perrya, B.J. Superb, D.P. GalloglyaaDepartment of Psychology, Uni6ersity of Texas at Austin, Austin, TX78712, USAbDepartment of Electrical Engineering and Computer Science, Uni6ersity of Illinois at Chicago, Chicago, IL60607, USAReceived 29 March 2000; received in revised form 13 October 2000AbstractThe human brain manages to correctly interpret almost every visual image it receives from the environment. Underlying thisability are contour grouping mechanisms that appropriately link local edge elements into global contours. Although a general viewof how the brain achieves effective contour grouping has emerged, there have been a number of different specific proposals andfew successes at quantitatively predicting performance. These previous proposals have been developed largely by intuition andcomputational trial and error. A more principled approach is to begin with an examination of the statistical properties of contoursthat exist in natural images, because it is these statistics that drove the evolution of the grouping mechanisms. Here we reportmeasurements of both absolute and Bayesian edge co-occurrence statistics in natural images, as well as human performance fordetecting natural-shaped contours in complex backgrounds. We find that contour detection performance is quantitativelypredicted by a local grouping rule derived directly from the co-occurrence statistics, in combination with a very simple integrationrule (a transitivity rule) that links the locally grouped contour elements into longer contours. © 2001 Elsevier Science Ltd. Allrights reserved.Keywords:Contour perception; Form perception; Grouping; Natural imageswww.elsevier.com/locate/visres1. IntroductionDuring the past decade, a number of psychophysical(Kellman & Shipley, 1991; Field, Hayes, & Hess, 1993;McIlhagga & Mullen, 1996; Dakin & Hess, 1998),neurophysiological (Bosking, Zhang, Schofield, & Fitz-patrick, 1997; Kapadia, Westheimer, & Gilbert, 1999),and computational (Sha’ashua & Ullman, 1988; Parent& Zucker, 1989; Yen & Finkel, 1998) studies havehelped develop the 75-year-old Gestalt notion of ‘goodcontinuation’ (Wertheimer, 1958) into rigorous ac-counts of contour grouping. The general view that hasemerged can be summarized using three processingstages. In the first stage, local contour elements areextracted using spatial filtering like that observed inprimary visual cortex. In the second stage, ‘bindingstrengths’ are formed between local contour elements.These binding strengths are often described by a localgrouping function (an ‘association field’ according Fieldet al., 1993), which specifies binding strength as afunction of differences in the position, orientation, con-trast, and so on, of the contour elements. In the thirdstage, an integration process uses the local bindingstrengths to group the local elements into globalcontours.If the grouping mechanisms evolved to optimize con-tour perception in the natural environment then theshape of the local grouping function should be closelyrelated to the statistical co-occurrence of edge elementsin natural images, and explanations of contour group-ing that are based upon edge co-occurrence shouldbetter account for human ability to detect naturalcontours in complex backgrounds. To test these predic-tions we measured the probabilities of all possiblegeometrical relationships between pairs of edge ele-ments extracted from natural images. We measuredboth the absolute co-occurrence statistics, which in prin-ciple could be ‘learned’ directly from the images with-out feedback, and the Bayesian co-occurrence statistics,* Corresponding author. Fax: + 1-512-4717356.E-mail address:[email protected] (W.S. Geisler).0042-6989/01/$ - see front matter © 2001 Elsevier Science Ltd. All rights reserved.PII: S0042-6989(00)00277-7W.S. Geisler et al./Vision Research41 (2001) 711– 724712which would require feedback via interaction with theenvironment.1Then, we derived a local grouping func-tion directly from the edge co-occurrence probabilities,combined this local grouping function with a verysimple integration rule, and compared the predictionsto parametric detection data for naturalistic contours.2. Absolute edge co-occurrence statistics2.1. Methods2.1.1. Edge extractionThe first step in measuring the edge co-occurrencestatistics is to extract edge elements from natural im-ages. Edge elements were extracted from 20 representa-tive natural images by measuring local contrast energyacross orientation at all potential edge locations.Twenty images were selected to represent a wide rangeof natural terrain (see Fig. 1). Each image was con-verted to 8-bit gray scale using Adobe PhotoShop, andthen windowed with a circular aperture to a diameter of480 pixels.Significant edge elements were then extracted fromeach image using a method that mimics the responsecharacteristics of neurons in primary visual cortex.First, potential edge locations were identified by filter-ing the image with a non-oriented log Gabor functionhaving a spatial-frequency bandwidth of 1.5 octaves,and a peak spatial frequency of 0.1 cycles/pixel.2Eachzero-crossing pixel in the filtered image (within a radiusof 216 pixels of the image center) was regarded as apotential edge element.Second, the original image was filtered with orientedlog Gabor functions having a spatial frequency band-width of 1.5 octaves and an orientation bandwidth of40°, which are the average values reported for primatevisual cortex (Geisler & Albrecht, 1997). The peakspatial frequency of the log Gabor filters was set to 0.1cycles/pixel. This value was picked so that the spatialscale of the filter kernels (‘receptive fields’) werematched to the size of the contour elements used in thepsychophysical experiments. Filtered images were ob-tained for log Gabor functions in both sine and cosinephase, at every 10° of orientation. The sine and cosinefiltered images were squared and summed to obtain thecontrast energy at each orientation for each zero-cross-ing pixel. The normalized contrast energy at each orien-tation was then obtained by dividing the contrastenergy at that orientation by the sum of the contrastenergies across all orientations, plus a constant. (Theconstant was set so that the half-saturation contrast ofthe response to an optimal sine wave grating corre-sponds to the average in monkey