Natural Scene Categorization: from Humans to ComputersWhat is “gist”?What is “gist”?What is “gist”?What is “gist”?What is “gist”?What is “gist”?What do people see in a glance?Scene levelObject level(Social) Events Analogy to documentsmodel distance based on theme distributionSummaryreferencesNatural Scene Categorization:from Humans to ComputersLi Fei-FeiBeckman Institute, ECE, Psychologyhttp://visionlab.ece.uiuc.eduMIT Suns06 2006.02.17 Li Fei-Fei, UIUC#1: natural scene categorization entails little attention (Rufin VanRullen, Christof Koch, Pietro Perona)#2: what can we perceive within a glance of a scene – a working definition for ‘gist’(Asha Iyer, Christof Koch, Pietro Perona)#3: local patches, and some intermediate level information – a hierarchical Bayesian algorithm for natural scene categorization (Pietro Perona)An outdoor s cene, I think. reminded me a a city... like walki ngin a park in n ew york or som ethin g. th ere seemed to be trees and a road and then this large skyscr aper in the backgroun d.Please type your description here:MIT Suns06 2006.02.17 Li Fei-Fei, UIUC• #1: natural scene categorization entails little attentionReference: Li et al. 2002; Fei-Fei et al. 2005Thorpe, et al 1996150 ms !!150 ms !!Thorpe, et al 1996Our question1. How critical is attention in natural scene recognition? 2. How does this compare to other recognition tasks?attentional loadmorelessLi et al. 2002animals vehicleTsynthetic stimuliattentional loadmorelessLi et al. 2002our finding…Without color…Effect of “meaningful” categoryMIT Suns06 2006.02.17 Li Fei-Fei, UIUC• #2: what can we perceive within a glance of a scene – a working definition for ‘gist’Reference: Fei-Fei et al. submittedWhat is “gist”?• sensory data, e.g. “color”, “size”, etc.Wolfe, 1998; Goffaux et al. 2005What is “gist”?• sensory data, e.g. “color”, “size”, etc.• “inventory of some of the objects (and textures)”Wolfe, 1998water sand skypalmtreeWhat is “gist”?• sensory data, e.g. “color”, “size”, etc.• “inventory of some of the objects”• “some relationships between objects”Wolfe, 1998What is “gist”?• sensory data, e.g. “color”, “size”, etc.• “inventory of some of the objects”• “some relationships between objects”•“layout”Biederman et al. 1987, Wolfe, 1998What is “gist”?• sensory data, e.g. “color”, “size”, etc.• “inventory of some of the objects”• “some relationships between objects”•“layout”• “stuffness”Wolfe, 1998What is “gist”?• sensory data, e.g. “color”, “size”, etc.• “inventory of some of the objects”• “some relationships between objects”•“layout”• “stuffness”• scene categoryWolfe, 1998beachWhat do people see in a glance?timeImage onset: t = 0 msecMask onset: t = SOA1 of 7 possible SOA’s (msec):27, 40, 53, 67, 80, 120, 500Subject types freely what he/she saw in the imageStage I: Collect Image Description--- Illustration of 1 TrialAn outdoor scene, I think. reminded me a a city... like walkingin a park in new yorkor something. there seemed to be trees and a road and then this large skyscraper in the background.Please type your description here:PT = 27msPT = 40msPT = 67msPT = 500msFei-Fei et al. submittedThe treeScene levelObject level(Social) EventsMIT Suns06 2006.02.17 Li Fei-Fei, UIUC• #3: local patches, and some intermediate level information – a hierarchical Bayesian algorithm for natural scene categorization Reference: Fei-Fei et al. CVPR 2005Jain, Zhang et al. (1998)••global cues: colors, textures, etc.global cues: colors, textures, etc.Szummer et al. (1998)••global cues: frequencyglobal cues: frequencyOliva, Torralba (1999, 2001)opennessroughnessexpansionopennessruggedness roughnessVogel & Schiele (2004)local patch based idea• local patch based• intermediate level themes within scenesforest suburb inside of cityFei-Fei & Perona (CVPR 2005)Our intuitionsOur intuitionsOur intuitionsOur intuitions• local patch based• intermediate level themesFei-Fei & Perona (CVPR 2005)• weakly supervised−no human annotation of local patches and intermediate level themesImageImageBag of Bag of ‘‘wordswords’’Analogy to documentsAnalogy to documentsOf all the sensory impressions proceeding to the brain, the visual experiences are the dominant ones. Our perception of the world around us is based essentially on the messages that reach the brain from our eyes. For a long time it was thought that the retinal image was transmitted point by point to visual centers in the brain; the cerebral cortex was a movie screen, so to speak, upon which the image in the eye was projected. Through the discoveries of Hubel and Wiesel we now know that behind the origin of the visual perception in the brain there is a considerably more complicated course of events. By following the visual impulses along their path to the various cell layers of the optical cortex, Hubel and Wiesel have been able to demonstrate that the message about the image falling on the retina undergoes a step-wise analysis in a system of nerve cells stored in columns. In this system each cell has its specific function and is responsible for a specific detail in the pattern of the retinal image.sensory, brain, visual, perception, retinal, cerebral cortex,eye, cell, optical nerve, imageHubel, WieselChina is forecasting a trade surplus of $90bn (£51bn) to $100bn this year, a threefold increase on 2004's $32bn. The Commerce Ministry said the surplus would be created by a predicted 30% jump in exports to $750bn, compared with a 18% rise in imports to $660bn. The figures are likely to further annoy the US, which has long argued that China's exports are unfairly helped by a deliberately undervalued yuan. Beijing agrees the surplus is too high, but says the yuan is only one factor. Bank of China governor Zhou Xiaochuan said the country also needed to do more to boost domestic demand so more goods stayed within the country. China increased the value of the yuan against the dollar by 2.1% in July and permitted it to trade within a narrow band, but the US wants the yuan to be allowed to trade freely. However, Beijing has made it clear that it will take its time and tread carefully before allowing the yuan to rise further in value.China, trade, surplus, commerce, exports, imports, US, yuan, bank, domestic, foreign, increase, trade,