Slide 1Last LectureThis weekToday: Three papers on computational models of context:Why is detection hard?Is local information enough?Slide 7Is local information even enough?Is local information even enough?Slide 10The multiple personalities of a blobThe multiple personalities of a blobSlide 13Slide 14Slide 15Look-Alikes by Joan SteinerLook-Alikes by Joan SteinerLook-Alikes by Joan SteinerThe context challengeWhat are the hidden objects?What are the hidden objects?The importance of contextSlide 23Object representationsObject representationsPrevious work on contextPrevious work on contextPrevious work on contextPrevious work on contextGraphical models for image labelingPrevious work on contextOutline of this talkImage databaseWhich objects are important?Object representationLearning local features (intrinsic object features)Object local featuresResults with local featuresResults with local featuresResults with local featuresGlobal context: location primingObject global featuresObject global featuresCar detection with global featuresCombining global and localClustering of objects with local and global feature sharingOutline of this talkAdding correlations between objectsLearning in CRFsSequentially learning the structureSequentially learning the structureCar detectionCar detectionScreen/keyboard/mouseCascadeCascadeCascadeSlide 58Slide 59Understanding an ImageToday: Local and IndependentWhat the Detector SeesLocal Object DetectionWork in ContextReal Relationships are 3DRecent Work in 3DObjects and ScenesContribution of this PaperObject SupportSurface EstimationObject Size in the ImageObject Size ↔ Camera ViewpointObject Size ↔ Camera ViewpointObject Size ↔ Camera ViewpointObject Size ↔ Camera ViewpointObject Size ↔ Camera ViewpointObject Size ↔ Camera ViewpointWhat does surface and viewpoint say about objects?What does surface and viewpoint say about objects?Scene Parts Are All InterconnectedInput to Our AlgorithmScene Parts Are All InterconnectedOur Approximate ModelInference over Tree Easy with BPViewpoint estimationObject detectionExperiments on LabelMe DatasetEach piece of evidence improves performanceCan be used with any detector that outputs confidencesAccurate Horizon EstimationQualitative ResultsQualitative ResultsQualitative ResultsQualitative ResultsSummary & Future WorkConclusionSlide 97Learning Spatial Context: Using stuff to find thingsThings vs. StuffFinding ThingsOutlineSatellite Detection ExampleError AnalysisTypes of ContextTypes of ContextOutlineThingsThingsStuffRelationshipsThe TAS ModelUnrolled ModelLearning the ParametersLearned Satellite ClustersWhich Relationships to Use?Learning the RelationshipsInferenceOutlineBase Detector - HOGResults - SatelliteResults - SatellitePASCAL VOC ChallengeBase Detector Error AnalysisDiscovered Context - BicyclesTAS Results – BicyclesResults – VOC 2005Results – VOC 2006ConclusionsToday: Three papers on computational models of context:Slide 130C280, Computer VisionProf. Trevor [email protected] 16: Recognition in ContextLast Lecture•Naïve-Bayes Nearest Neighbor (Irani)•ISM (Liebe)•Constellation Models (Fergus)•Transformed LDA Models (Sudderth)•3-D view models (Saravese)This week•Two last topics in recognition:–Context–ArticulationToday: Three papers on computational models of context:•A. Torralba, K. P. Murphy, and W. T. Freeman, "Contextual models for object detection using boosted random fields," in Advances in Neural Information Processing Systems 17 (NIPS), 2005. •D. Hoiem, A. A. Efros, and M. Hebert, "Putting objects in perspective," in Computer Vision and Pattern Recognition, 2006•G. Heitz and D. Koller, "Learning spatial context: Using stuff to find things," in ECCV 2008, pp. 30-43.Why is detection hard?xscaley1,000,000 images/dayPlus, we want to do this for ~ 1000 objects10,000 patches/object/imagetimeIs local information enough?Slide credit: A. TorralbaWith hundreds of categoriesroadtablechairkeyboardtablecarroadIf we have 1000 categories (detectors), and each detector produces 1 fa every 10 images, we will have 100 false alarms per image… pretty much garbage… Slide credit: A. TorralbaIs local information even enough?Slide credit: A. TorralbaIs local information even enough?DistanceInformationLocal featuresContextual featuresSlide credit: A. TorralbaWe know there is a keyboard present in this scene even if we cannot see it clearly.We know there is no keyboard present in this scene… even if there is one indeed.The system does not care about the scene, but we do…Slide credit: A. TorralbaThe multiple personalities of a blobSlide credit: A. TorralbaThe multiple personalities of a blobSlide credit: A. TorralbaSlide credit: A. TorralbaSlide credit: A. TorralbaSlide credit: A. TorralbaLook-Alikes by Joan SteinerSlide credit: A. TorralbaLook-Alikes by Joan SteinerSlide credit: A. TorralbaLook-Alikes by Joan SteinerSlide credit: A. TorralbaThe context challengeHow far can you go without using an object detector?Slide credit: A. Torralba21What are the hidden objects?Slide credit: A. TorralbaWhat are the hidden objects?Chance ~ 1/30000Slide credit: A. TorralbaThe importance of context•Cognitive psychology–Palmer 1975 –Biederman 1981–…•Computer vision–Noton and Stark (1971)–Hanson and Riseman (1978)–Barrow & Tenenbaum (1978) –Ohta, kanade, Skai (1978)–Haralick (1983)–Strat and Fischler (1991)–Bobick and Pinhanez (1995)–Campbell et al (1997)Slide credit: A. TorralbaMulticlass object detection and context modeling Antonio TorralbaIn collaboration with Kevin P. Murphy and William T. FreemanObject representationsPartsGlobal appearanceObject sizeInside the object(intrinsic features)PixelsAgarwal & Roth, (02), Moghaddam, Pentland (97), Turk, Pentland (91),Vidal-Naquet, Ullman, (03)Heisele, et al, (01), Agarwal & Roth, (02), Kremp, Geman, Amit (02), Dorko, Schmid, (03)Fergus, Perona, Zisserman (03), Fei Fei, Fergus, Perona, (03), Schneiderman, Kanade (00), Lowe (99)Etc.Object representationsPartsGlobal appearanceLocal contextGlobal contextObject sizeInside the object(intrinsic features)Outside the object(contextual features)PixelsKruppa & Shiele, (03), Fink & Perona (03)Carbonetto, Freitas, Barnard (03), Kumar, Hebert, (03)He, Zemel, Carreira-Perpinan (04), Moore, Essa, Monson, Hayes (99)Strat & Fischler (91), Murphy, Torralba & Freeman (03)Agarwal & Roth, (02), Moghaddam, Pentland (97), Turk, Pentland (91),Vidal-Naquet, Ullman, (03)Heisele,