Journal of Vision 2008 8 3 3 1 15 http journalofvision org 8 3 3 1 Interesting objects are visually salient Department of Computer Science University of Southern California Los Angeles CA USA Lior Elazary Department of Computer Science and Neuroscience Graduate Program University of Southern California Los Angeles CA USA Laurent Itti How do we decide which objects in a visual scene are more interesting While intuition may point toward high level object recognition and cognitive processes here we investigate the contributions of a much simpler process low level visual saliency We used the LabelMe database 24 863 photographs with 74 454 manually outlined objects to evaluate how often interesting objects were among the few most salient locations predicted by a computational model of bottom up attention In 43 of all images the model s predicted most salient location falls within a labeled region chance 21 Furthermore in 76 of the images chance 43 one or more of the top three salient locations fell on an outlined object with performance leveling off after six predicted locations The bottom up attention model has neither notion of object nor notion of semantic relevance Hence our results indicate that selecting interesting objects in a scene is largely constrained by low level visual properties rather than solely determined by higher cognitive processes Keywords attention awareness sensory integration objects scene understanding Citation Elazary L Itti L 2008 Interesting objects are visually salient Journal of Vision 8 3 3 1 15 http journalofvision org 8 3 3 doi 10 1167 8 3 3 Introduction Being able to identify interesting regions or objects in our cluttered visual environment is key to animal survival be it to locate possible prey mates predators navigation landmarks tools or food Yet very little is known of the computational neural mechanisms that underlie the behavioral selection of interesting objects in our visual world Is it that we first have to attend to and select a number of candidate visual locations then recognize the identity as well as a number of properties of each candidate and finally evaluate these against current behavioral goals intentions and preferences so as to decide whether that object was interesting or not Navalpakkam Itti 2005 Rensink 2000 Here we show that the first phase of such seemingly complicated and time consuming putative processVattentional selection based on intrinsic visual saliencyValready is a strong predictor of which regions in digital photographs were labeled by human observers as potentially interesting objects Focal visual attention has long been known to be a necessary first step in locating potentially interesting elements in a scene Itti Koch 2001 James 1890 Indeed unless attention is first directed toward a particular scene element much of its attributes and even possibly its very existence will remain unnoticed as has been vividly demonstrated by studies of change blindness doi 1 0 11 67 8 3 3 and inattentional blindness Mack Rock 1998 O Regan Rensink Clark 1999 Hence we hypothesized that regions or objects which human observers would find more interesting should also attract attention that is be visually salient Itti Koch Niebur 1998 Koch Ullman 1985 In this paper we define interesting objects or image regions as those which among all items present in a digital photograph people choose to label when given a fairly unconstrained image annotation task details below The assumption that people would choose to label interesting objects comes simply from the fact that there is some motivation for people to label one region whether being an object or not over another Early work interested in characterizing what may attract attention toward potentially interesting objects in scenes has suggested that changes in illumination on the retina is a particularly effective cue Franconeri Hollingworth Simons 2005 Jonides Yantis 1988 Yantis Jonides 1996 Indeed abrupt luminance changes are typically observed when a new object appears in the scene hence detecting such low level physical changes using luminance tuned visual neurons would often quite effectively guide attention toward interesting novel objects Kahneman Treisman Gibbs 1992 Other research suggests that sudden changes in color are also effective in attracting attention Snowden 2002 Turatto Galfano 2001 although this has been more largely debated Folk Annett 1994 Franconeri Simons 2003 Jonides Received April 20 2007 published March 7 2008 ISSN 1534 7362 ARVO Journal of Vision 2008 8 3 3 1 15 Elazary Itti Yantis 1988 Theeuwes 1995 Other influences also are tied to the behavioral task which an observer may be engaged in for example a search task Quinlan Humphreys 1987 Treisman Sato 1990 Wolfe Cave Franzel 1989 Indeed the effectiveness of simple bottom up information like color and illumination in attracting attention can be modulated by task influences to yield complex search patterns Desimone Duncan 1995 Evans Treisman 2005 Theeuwes 1994 Treisman Sato 1990 Underwood Foulsham 2006 Wolfe 1994 However the relative strength of contributions from bottom up information e g salience versus top down information e g relevance to a task in determining what people find interesting remains largely unknown Henderson 2003 Possibly when no specific search target no search task and no particular time or other constraint are specified to an observer bottom up information might play a predominant role in guiding attention toward potential generically interesting targets Itti 2005 Under such conditions e g under free viewing bottomup information could provide a strong indication of what people might find interesting in a given scene We used a computational model to compute saliency maps in digital photographs and to test the extent to which saliency at a given image location indicates how interesting that location may be to human observers Previous human eye tracking studies have shown that saliency is a strong predictor of attention and gaze allocation during Figure 1 Example of scenes and associated objects outlines 2 free viewing both in static images Parkhurst Law Niebur 2002 Tatler Baddeley Gilchrist 2005 Underwood Foulsham 2006 and in natural video stimuli Itti 2005 However it has not been shown under natural viewing conditions whether a visual location that is attracting the gaze is also being judged as interesting Intuitively it is possible that attentional selection based on low level visual saliency
View Full Document
Unlocking...