New version page

UT PSY 394U - Interesting objects are visually salient

Documents in this Course


6 pages

Load more
Upgrade to remove ads
Upgrade to remove ads
Unformatted text preview:

Interesting objects are visually salientDepartment of Computer Science,University of Southern California,Los Angeles, CA, USALior ElazaryDepartment of Computer Science,and Neuroscience Graduate Program,University of Southern California,Los Angeles, CA, USALaurent IttiHow do we decide which objects in a visual scene are more interesting? While intuition may point toward high-level objectrecognition and cognitive processes, here we investigate the contributions of a much simpler process, low-level visualsaliency. We used the LabelMe database (24,863 photographs with 74,454 manually outlined objects) to evaluat e how ofteninteresting objects were among the few most salient locations predicted by a computational model of bottom-up attention. In43% of all images the model’s predicted most salient location falls withi n a labeled region (chance 21%). Furthermore, in76% of the images (chance 43%), one or more of the top three salient locations fell on an outlined object, with performanceleveling off after six predicted locations. The bottom-up attention model has neither notion of object nor notion of semanticrelevance. Hence, our results indicate that selecting interesting objects in a scene is largely constrained by low-level visualproperties rather than solely determined by higher cognitive processes.Keywords: attention, awareness, sensory integration, objects, scene understandingCitation: Elazary, L., & Itti, L. (2008). Interesting objects are visually salient. Journal of Vision, 8(3):3, 1–15,, doi:10.1167/8.3.3.IntroductionBeing able to identify interesting regions or objects inour cluttered visual environment is key to animal survival,be it to locate possible prey, mates, predators, navigationlandmarks, tools, or food. Yet, very little is known of thecomputational neural mechanisms that underlie the behav-ioral selection of interesting objects in our visual world. Isit that we first have to attend to and select a number ofcandidate visual locations, then recognize the identity aswell as a number of properties of each candidate, andfinally evaluate these against current behavioral goals,intentions, and preferences, so as to decide whether thatobject was interesting or not (Navalpakkam & Itti, 2005;Rensink, 2000)? Here we show that the first phase of suchseemingly complicated and time-consuming putativeprocessVattentional selection based on intrinsic visualsaliencyValready is a strong predictor of which regions indigital photographs were labeled by human observers aspotentially interesting objects.Focal visual attention has long been known to be anecessary first step in locating potentially interestingelements in a scene (Itti & Koch, 2001; James, 1890).Indeed, unless attention is first directed toward a partic-ular scene element, much of its attributes and evenpossibly its very existence will remain unnoticed, as hasbeen vividly demonstrated by studies of change blindnessand inattentional blindness (Mack & Rock, 1998;O’Regan, Rensink, & Clark, 1999). Hence, we hypothe-sized that regions or objects which human observerswould find more interesting should also attract attention,that is, be visually salient (Itti, Koch, & Niebur, 1998;Koch & Ullman, 1985). In this paper, we define interest-ing objects or image regions as those which, among allitems present in a digital photograph, people choose tolabel when given a fairly unconstrained image annotationtask (details below). The assumption that people wouldchoose to label interesting objects comes simply from thefact that there is some motivation for people to label oneregion (whether being an object or not) over another.Early work interested in characterizing what may attractattention toward potentially interesting objects in sceneshas suggested that changes in illumination on the retina isa particularly effective cue (Franconeri, Hollingworth, &Simons, 2005; Jonides & Yantis, 1988; Yantis & Jonides,1996). Indeed, abrupt luminance changes are typicallyobserved when a new object appears in the scene; hence,detecting such low-level physical changes using lumi-nance-tuned visual neurons would often quite effectivelyguide attention toward interesting novel objects (Kahneman,Treisman, & Gibbs, 1992). Other research suggests thatsudden changes in color are also effective in attractingattention (Snowden, 2002; Turatto & Galfano, 2001),although this has been more largely debated (Folk &Annett, 1994; Franconeri & Simons, 2003; Jonides &Journal of Vision (2008) 8(3):3, 1–15 1doi: 10.1167/8.3.3 ISSN 1534-7362 * ARVOReceived April 20, 2007; published March 7, 2008Yantis, 1988; Theeuwes, 1995). Other influences also aretied to the behavioral task which an observer may beengaged in, for example, a search task (Quinlan &Humphreys, 1987; Treisman & Sato, 1990; Wolfe, Cave,& Franzel, 1989). Indeed, the effectiveness of simplebottom-up information, like color and illumination, inattracting attention can be modulated by task influences toyield complex search patterns (Desimone & Duncan, 1995;Evans & Treisman, 2005; Theeuwes, 1994; Treisman &Sato, 1990; Underwood & Foulsham, 2006; Wolfe, 1994).However, the relative strength of c ontributi ons frombottom-up information (e.g., salience) versus top-downinformation (e.g., relevance to a task) in determiningwhat people find interesting remains largely unknown(Henderson, 2003). Possibly, when no specific searchtarget, no search task, and no particular time or otherconstraint are specified to an observer, bottom-up infor-mation might play a predominant role in guiding attentiontoward potential generically interesting targets (Itti, 2005).Under such conditions (e.g., under free viewing), bottom-up information could provide a strong indication of whatpeople might find interesting in a given scene.We used a computational model to compute saliencymaps in digital photographs and to test the extent to whichsaliency at a given image location indicates how interest-ing that location may be to human observers. Previoushuman eye-tracking studies have shown that saliency is astrong predictor of attention and gaze allocation duringfree viewing, both in static images (Parkhurst, Law, &Niebur, 2002; Tatler, Baddeley, & Gilchrist, 2005;Underwood & Foulsham, 2006) and in natural videostimuli (Itti, 2005). However, it has not been shown, undernatural viewing conditions, whether a visual location thatis attracting the gaze is also being judged as

View Full Document
Download Interesting objects are visually salient
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...

Join to view Interesting objects are visually salient and access 3M+ class-specific study document.

We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Interesting objects are visually salient 2 2 and access 3M+ class-specific study document.


By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?