Biederman’s Recognition-by-Components (RBC) Theory of Image RecognitionVery rough overview of RBCVery rough overview of RBCKey questions:What’s the relevant visual input?Line layouts!Slide Number 7Slide Number 8Slide Number 9Diagnostic colors?Slide Number 11Key questionsKey questionsHow is the visual input segmented into parts?Slide Number 15Slide Number 16Slide Number 17Concavities arise where convex volumes are joined (Transversality Principle) Slide Number 19Key questionsWhat are the parts and how are they recognized?ConstraintsSlide Number 23Slide Number 24Slide Number 25Slide Number 26 Use non-accidental features instead of metric featuresSlide Number 28Then define parts in terms of those non-accidental featuresSlide Number 30…Step backCollinearitySlide Number 33Collinearity distinguished from CurvatureSlide Number 35Symmetry and ParallelismSlide Number 37Slide Number 38Slide Number 39Slide Number 40CoterminationSlide Number 42Slide Number 43Slide Number 44Slide Number 45Slide Number 46Slide Number 47Slide Number 48Slide Number 49GeonsSlide Number 51Slide Number 52How should we think of Geons?Bare feature n-tuples?Slide Number 55Slide Number 56Slide Number 57Slide Number 58Slide Number 59Slide Number 60Slide Number 61How many Geons are there?Slide Number 63Slide Number 64Slide Number 65Is this a problem?Slide Number 67Key questionsHow do geons combine?Slide Number 70Slide Number 71Slide Number 72Slide Number 73Slide Number 74Slide Number 75How much can we represent with geons in combination?Slide Number 77Homework:Key questionsSlide Number 80PredictionsSlide Number 82Slide Number 83Slide Number 84Experiment 1Slide Number 86Slide Number 87Slide Number 88Slide Number 89Slide Number 90Summary Experiment 1Experiment 2Slide Number 93Slide Number 94Slide Number 95Slide Number 96Slide Number 97Summary Experiment 2Experiment 3Slide Number 100Slide Number 101Slide Number 102Experiment 3 summaryExperiment 4Slide Number 105Slide Number 106Slide Number 107Slide Number 108Summary experiment 4Further observationsSlide Number 111Slide Number 112Slide Number 113Further observationsSlide Number 115Slide Number 116Further observationsSlide Number 118Slide Number 119Slide Number 120Further observationsFurther observations Slide Number 123Biederman and Bar (1999)Slide Number 125Slide Number 126Slide Number 127Slide Number 128Slide Number 129Slide Number 130Slide Number 131SummaryVogels et al. 2001backgroundSlide Number 135Slide Number 136Slide Number 137Kayaert et al. (2003) Slide Number 139Kayaert, Biederman, Op De Beek, Vogels (2005)Slide Number 141Kayaert, Biederman, Vogels (2005)Slide Number 143Hayworth and Biederman (2005)Slide Number 145Slide Number 146ConclusionReplies on behalf of BiedermanPaper clipsLogothetis et al. 1995. Evidence for IT neurons coding “blurred templates” not geometric featuresSlide Number 151Slide Number 152“We can look at the zig-zag horizontal brace as a texture region or zoom in and interpret it as a series of connected blocks” (Biederman)Regarding objects like cork-screws: “those regions are represented through the statistical processing that characterizes their texture.” not in terms of volumetric components unless we “zoom in”. (Biederman)Slide Number 155AmoebasSlide Number 157Biederman’s Recognition-by- Components (RBC) Theory of Image RecognitionPhilipp E. KoralusCos 598/Psy 594Prof. Fei-Fei LiSpring 2008Very rough overview of RBC¾ Visual input segmented into components¾ Components are recognized as falling into different categories of geons¾ Recognition memory coded in terms of geonsand how they are combinedVery rough overview of RBC• It’s primarily a theory of “first shot” recognition of novel and familiar objects• Predicts recognition robustness insofar as components still recognizableKey questions:1. What is the relevant visual input to first shot recognition?2. How is the visual input segmented into parts?3. What are the parts and how are they recognized?4. How do the parts combine?What’s the relevant visual input?Line layouts!• We can recognize objects rapidly and normally even if reduced to line drawings.• Kourtzi and Kanwisher (2000). Adaptation of fMRI BOLD signal in LOC maintained if image changes from gray- level photo to line drawing of the same object.QuickTime™ and aTIFF (LZW) decompressorare needed to see this picture.• Biederman and Ju (1986). Recognition RTs in naming task virtually the same using line drawings and full color photographs.• Images presented by color photography were 11ms faster than the corresponding drawing but had a 3.9% higher error rate.Diagnostic colors?QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.• Objects with a diagnostic color did not enjoy any advantage when they were displayed as color slides compared with their line drawing versions.• Not even in a “tell me if you see a banana among the following slides” task!Key questions1.What is the relevant visual input to first shot recognition?Key questions1. What is the relevant visual input to first shot recognition?Æ Line layouts are sufficientHow is the visual input segmented into parts?QuickTime™ and aTIFF (LZW) decompressorare needed to see this picture.• We tend to segment at regions of sharp concavity• People tend to agree on what the natural components of an object areQuiTIFF (Uncompare neededQuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.Concavities arise where convex volumes are joined (Transversality Principle)QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.• Good continuation is a statistically powerful cue to determining object boundaries in natural scenes. Elder and Goldberg (2002)• Sharp convexities break good continuation.Key questions1. What is the relevant visual input to first shot recognition?Æ Line layouts are sufficient2. How is the visual input segmented in parts?Æ Segment parts at sharpconvexitiesWhat are the parts and how are they recognized?Constraints• We are bad at making absolute judgments in length, tilt or curvature (Beck, Prazdny, and Rosenfeld 1983; Fildes & Triggs, 1985; Garner, 1962; Miller 1956; Virsu, 1971).• Our memory for shape shows tendency for regularization (Woodworth 1938).Errors in reproduction:Slight deviations from symmetrical or regular figures.Alternatively, irregularities are accentuated as regular subparts• We tend to see