Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 23: Object RecognitionInitialize‡Spell check offOff@General::spell1D;OutlineLast timeScience writingTodayTasksObject recognitionRole of geometric modeling in theories of object recognitionHigh-level vision, visual tasksIntermediate-level visionGeneric, global organizational processesDomain overlap, occlusionSurface grouping, selectionGestalt principlesCue integrationCooperative computationAttentionHigh-level visionFunctional tasksObject recognitionentry-level, subordinate-levelObject-object relationsScene recognitionSpatial layoutViewer-object relationsObject manipulationreach & graspHeadingTask dependency: explicit (primary) and generic (secondary, nuisance) variablesI=f(shape, material, articulation,viewpoint,relative position, illumination)2 23.ObjectRecognition.nb‡Task: Object Recognition‡Task: Absolute depth (e.g. for reaching)23.ObjectRecognition.nb 3‡Task: graspProblem: all the scene variables contribute to the variations in the image‡Preview of Mathematica application below Shape-based object recognitionestimate geometrical shape (primary variables)discount sources of image variation not having to do with shape (secondary variables)e.g. integrating out geometrical variables such as translation, rotation, and scale to estimate shape for object recognitionObject recognitionSources of image variationWe'll work from lower to higher levels of object abstraction‡Variation within subordinate-level category (subordinate level, e.g. mallard, Doberman, Braeburn)illuminationlevel, direction, source arrangement, shadows,spectral contentviewscaletranslation2D & 3D rotationarticulationnon-rigid, e.g. joints, hinges, facial expression, hair, clothbackground (segmentation)bounding contour/edge variationocclusion (segmentation)4 23.ObjectRecognition.nbilluminationlevel, direction, source arrangement, shadows,spectral contentviewscaletranslation2D & 3D rotationarticulationnon-rigid, e.g. joints, hinges, facial expression, hair, clothbackground (segmentation)bounding contour/edge variationocclusion (segmentation)‡Variation within basic-level category (e.g. duck, dog, chair, apple)"entry-level", "basic-level"structural relation invariance?‡Variation across super-ordinate category (e.g.bird, mammal, furniture, fruit )more cognitive than perceptual, non-pictorial‡Variation across contextball on tennis court vs. billiard tableBasic-level vs. subordinate-levelPsychological (Rosch et al.), Neuropsychological (Damasio and Damasio)temporal lobe lesions disrupt object recognitionfine-grain distinctions more easily disrupted than coarse-grain onese.g. Boswell patient~can't recognize faces of family, friends, unique objects, or unique places. Can assign names like face, house, car, appropriately. Also superordinate categories: "tool"prosopagnosicsfaces vs. subordinate-level?neural evidence for distinction? IT hypercolumns?‡Basic-level23.ObjectRecognition.nb 5‡Basic-levelShape-particularly critical -- but qualitative, rather than metric aspects important. E.g. geons and geon relations (Biederman).Material, perhaps for some natural classes?Issue of prototypes with a model for variation vs. parts.e.g. average image face, the most familiarprimingFragment-based methods or "features of intermediate complexity": Ullman, S., Vidal-Naquet, M., & Sali, E. (2002). Learning informative features for class categorization‡Subordinate-levelgeometric variations important for subordinate--e.g. sensitivity to configurational etc..material Prototypes -> what kind of model for variation?Problem: With only a discrete set of views, how does vision generalize to other views?Consider viewpointFeatures invariant to viewpoint change?Material (surface color, texture)2D image features correlated with 3D ShapeObject ensemble is importantE.g only red object amidst others | no need to process its shape Context is importantSmall red thing flying past the window.High "cue validity" for Cardinal‡Context can even over-ride local cues for identitySinha and Poggio, Nature 19966 23.ObjectRecognition.nbSee too: Cox, D., Meyers, E., & Sinha, P. (2004). Contextually evoked object-specific responses in human visual cortex. Science, 304(5667), 115-117.Getting a good image representationOverview of processes that seem to be necessary-- i.e.what we think we knowFor object recognition, the contributions due to the secondary or "generic variables", (e.g. illumination and viewpoint) need to be discounted, and object features such as shape and material need to be estimated. How?o Measurements of image information likely to belong to the object. This principle should constrain segmentation.problems with: specularities, cast shadows, attached shadows (from shading). edge detection is really noisy, and ambiguous as to cause, so what are these image "features"?maybe although noisy, edges are sufficiently reliable to determine object class?o "Sensor fusion" or cue integration to improve estimates of where object boundaries are located:combine stereo, motion, chromatic, luminance, etc..o Incorporate intermediate-level constraints to help to find object boundaries or "silhouettes". Gestalt principles of perceptual organizationMohan (1988); Zhu (1999); Geiger | figure/ground talksymmetry (Vetter et al., 1994)long smooth lines (David & Zucker, 1989l; Shashua & Ullman, 1988; Field and Hess, 1993)o "Cooperative computation" for object shape, reflectance and lighting.There is no single local cue to edge identity"intrinsic images" of Barrow and Tenenbaum to extractClark & Yuille (1990); Knill &Kersten, Kersten (1991); Kersten and Madarasmi (1995). Problem: Still no bottom-up procedure for perfect segmentation or edge-parsing.Solutions?: Domain-specific processing; class-specific recognition.Classification -> edge detection. E.g dalmation dog demonstration23.ObjectRecognition.nb 7For object recognition, the contributions due to the secondary or "generic variables", (e.g. illumination and viewpoint) need to be discounted, and object features such as shape and material need to be estimated. How?o Measurements of image information likely to belong to the object. This principle should constrain segmentation.problems with: specularities, cast shadows, attached shadows (from shading). edge detection is really noisy, and ambiguous as to cause, so what are these image "features"?maybe although noisy, edges are sufficiently reliable to determine object class?o "Sensor fusion" or cue