02/25/09 15-494 Cognitive Robotics 1Gestalt Perception15-494 Cognitive RoboticsDavid S. Touretzky &Ethan Tira-ThompsonCarnegie MellonSpring 200902/25/09 15-494 Cognitive Robotics 2What's a Gestalt?●German word meaning “whole” or “form”.●A complete pattern or configuration.●A configuration or pattern of elements so unified that... The whole is greater than the sum of its parts.●The term “sum,” in German ”Zusammenfassung,” means “summing up” or “synopsis”, not arithmetic sum.02/25/09 15-494 Cognitive Robotics 3Gestalt Perception●Describes perceptual phenomena and theoretical principles governing perceptual organization.–Perceptual grouping.–Figure-ground organization.–Frames of reference.–Pragnanz: tendency toward regular, ordered, stable, balanced states.●A holistic view of perception, rather than purely bottom-up.02/25/09 15-494 Cognitive Robotics 4Principles of Grouping●A: proximity●B: similar color●C: similar size●D: common fate●E: good continuation●F: closure●G: common region●H: element connectednessFrom S. Palmer, “Gestalt Perception”, MIT Encyclopedia of Cognitive Science02/25/09 15-494 Cognitive Robotics 5ProximitySimilarity02/25/09 15-494 Cognitive Robotics 6Good Continuation●Two smooth lines: a-b and c-d,that cross:●Not two sharp curves a-c and b-dmeeting at a point:02/25/09 15-494 Cognitive Robotics 7ClosureClosure and proximity can conflict:02/25/09 15-494 Cognitive Robotics 8Simulated Perceptual Grouping●K. R. Thorisson (1994) model of perceptual grouping:●How would you group these shapes?●What principles apply?02/25/09 15-494 Cognitive Robotics 9Thorisson ModelMultiple runs produce different results:02/25/09 15-494 Cognitive Robotics 10Thorisson (cont.)02/25/09 15-494 Cognitive Robotics 11Figure-Ground Principles●I: boundary belongs to the figure●J: surroundedness●K: size (figure is smaller)●L: figure is higher contrast●M: convexity●N: symmetry Not a quantitative theory: no way to predict how these principles combine or interact.From S. Palmer, “Gestalt Perception”, MIT Encyclopedia of Cognitive Science02/25/09 15-494 Cognitive Robotics 12Which Part is the Figure?●Smaller●Higher contrast●Surrounded●Contrast02/25/09 15-494 Cognitive Robotics 13Smallness●Figure tends to be smaller than ground.●The Rubin Vase:Vase is favored on the left; faces on the right.02/25/09 15-494 Cognitive Robotics 14Surroundedness●Areas surrounded by others tend to be seen as figure.●This can be misleading. What do you see below?02/25/09 15-494 Cognitive Robotics 15Symmetry●Is this two overlapping diamonds?●Or is it a diamond and two odd pieces?02/25/09 15-494 Cognitive Robotics 16Frame of Reference●O: induced motion (figure moves relative to reference frame, not vice versa)●P: rod-and-frame effectFrom S. Palmer, “Gestalt Perception”, MIT Encyclopedia of Cognitive Science02/25/09 15-494 Cognitive Robotics 17Other Organizational Phenomena●Q: amodal completion.●R: illusory contours●S: color scissionFrom S. Palmer, “Gestalt Perception”, MIT Encyclopedia of Cognitive Science02/25/09 15-494 Cognitive Robotics 18Kanizsa Triangle:Illusory Contours02/25/09 15-494 Cognitive Robotics 19Illusory ContoursEhrenstein Effect02/25/09 15-494 Cognitive Robotics 20Simple CurvesFrom L. Brassard, “The Visual World”02/25/09 15-494 Cognitive Robotics 21Implications for Robotics●Robots should “see” the world the way people do.●Recognize spatial relationships such as boundaries, contours, groups of things.●How do you “stand in line”?102/25/09 15-494 Cognitive Robotics 22Implications for Robotics●What's going on here?02/25/09 15-494 Cognitive Robotics 23Implications for Robotics●Where do you park the car?02/25/09 15-494 Cognitive Robotics 24Implications for Robotics●Where do you park the plane??02/25/09 15-494 Cognitive Robotics 25Why Was Parsing the Tic-Tac-Toe Board Difficult?●Grid of lines defined regions that weren't fully enclosed.●We had to induce our own “completion” effect to extract the board regions.●Working in camera space forced us to deal with perspective effects.02/25/09 15-494 Cognitive Robotics 26Can More Primitives Help?●Convex hull for defining regions:●Boundary detection and extrapolation:102/25/09 15-494 Cognitive Robotics 27Convex Hull in Tekkotsu●The convex hull operator is a crude approximation to gestalt completion, but only for convex shapes.●Defined in DualCoding::PolygonData:Shape<PolygonData> convexHull(const Sketch<bool> &sketch)●A polygon is a collection of edges and vertices (dual representation for efficiency).●Why isn't convex hull sufficient?–Some shapes aren't convex–Not a good representation of curved boundaries02/25/09 15-494 Cognitive Robotics 28Pragnanz (“Good Form”)Revisited●Early gestalt theorists (1930s)` suggested that holistic perception came from interacting electrical fields in the brain.–This has since been disproved.●More recently, recurrent neural networks such as the Hopfield network or Boltzmann machine have reproduced some aspects of holistic perception.–Perceptual principles act as sources of constraint on states of the network.–Network searches for “minimum energy” states that optimally satisfy the constraints.02/25/09 15-494 Cognitive Robotics 29Kienker et al.(1986)●Boltzmann machine doing figure-ground separation.●Edge units (triangles) support some neighboring figure units (squares) and inhibit others.●Opposite-orientation edges inhibit each other.02/25/09 15-494 Cognitive Robotics 30Edge Constraints●Activation propagates among edge units.●These local constraints will combine to produce a globally coherent solution.02/25/09 15-494 Cognitive Robotics 31Kienker et al.●Start with an outline.●Network probabilistically settles into a state in which figure is marked and ground is unmarked.02/25/09 15-494 Cognitive Robotics 32Spiral