1Approaches to Representing and Recognizing ObjectsVisual ClassificationCMSC 828J – David JacobsWhat the course is about• Visual Classification– Recognizing nouns and verbs from images.• This is one of the key problems of vision/cognition.– ~ half of cerebral cortex is vision.– Vision divides roughly into what and where.• This class is about what.2This is very hard• What is a class?– Ill-defined.– Tremendous variability.• And how do we relate images to objects or actions?• Current solutions grossly inadequate.So how do we have a class about this?1. Learn some fundamental things relevant to visual classification2. See how researchers have tried to apply these to visual classification.3Fundamental things• Lectures (but also discussion of some important papers).• Much of it mathematical and computational– Geometry of projection and invariance; PCA; shape spaces and shape matching; stochastic models of classes; learning theory.• But we draw from other fields too:– Philosophy: what is a class?– Biology: how does shape vary in nature?– Psychology: How do people do classification?Application of these ideas to visual classification• Read papers and discuss.• Shows how fundamental ideas can be used.• How math and computation interact.• Don’t solve big problem, but often useful for smaller problems.4Class Goal: Prepare us to solve problems of visual classification• Learn fundamental concepts important for vision.• Get us to think about what classification is.• Understand state-of-the-art attempts to solve it.Approaches to Visual Classification• Definitional: a class is defined by the presence or absence of properties (a point in feature space).• A class is a subspace of images.• Class is determined by similarity of images.– To prototypes or exemplars• Class represented by a generative model.• Class described by object parts and their relations.• Classes and generic learning.5A tour of the syllabusHow this might change• Probably way too much material.• Lectures may be longer than indicated.• I am open to suggestions about other papers or topics6Requirements (1)• Read papers before they are presented.• Paper Questions– On classes with paper discussions (all except first, last and midterm) you must provide an answer to a posted question.– Your answer should demonstrate knowledge of the papers and critical thought.– You may skip when you are giving a presentation and one other time.– 15% of grade.Requirements (2)• Presentations– For each presentation class, two teams of students will debate– Advocate one of two competing research directions. Examples:• 3D knowledge vs. 2D • Better descriptors vs. better ways to use them • What is the best direction for building a robot that can recognize household objects? • How can one represent complex classes of objects, such as chairs? Machine learning, manifolds, graphical models.– 15% of grade.7Requirements (3)• Midterm and Final– Will cover materials in lectures.– 50% of grade.Requirement (4)• Project: Implement two recognition algorithms.– Choose a data set. Examples:• Parkinson’s disease, PIE, face recognition with weight change, leaves, CalTech, PASCAL, …– Pick two algorithms. At least one non-trivial.– Implement and evaluate them8Your Background• Calculus, linear algebra, probability is essential.• Math that makes you really learn these topics is important.• Other math very helpful: geometry, stochastic processes, optimization.• Knowledge of vision may help a little.First Homework• Readings for Wednesday.• Review is due.9What is a category?Related Papers• Women, Fire and Dangerous Things by Lakoff, Chapters 1 and 2.• S. Laurence and E. Margolis, ``Concepts and Cognitive Science'', in Concepts edited by E. Margolis and S. Laurence, MIT Press.• L. Wittgenstein, Philosophical Investigations, sections 65-78.10Categorization is a critical issue in many disciplines• Linguistics – what do words mean• Psychology – All these issues are central, as in computer vision.• Philosophy – What do statements mean, what does it take for them to be true.Key questions a theory of categorization should answer• Acquisition – how is a category learnable.• Application – how do we decide on a category• Composition of categories– Big question in language and philosophy, since we combine categories so flexibly, and can understand a category we’ve never heard of before (blind, green fish).– Interesting in vision: big fish, red chair.• Is a category the sum of its parts?• Wooden spoon. Not just a spoon that is wooden, as most wooden spoons have different shapes. How do we combine information about spoons, wood, and wooden spoons.• What categories can occur? – In language, any category seems understandable. But some may be more natural. In vision, what categories are visually understood.• Others less relevant to us.11So, what is a category? What is a chair? A bird?Classical Theory• Categories have definitions – bachelor = unmarried adult male• Necessary and sufficient conditions• May be defined in terms of other concepts, but these ultimately bottom out in statements verified by our senses.– Ie, red is visually observable by reactions of cones to light.• Seems to offer answers to all core questions.• Part of a larger philosophical program.– A statement has meaning because it can be translated into logic,with verifiable atoms.• Held with little question for ~2,000 years.• Initial focus of AI, cognitive science: eg., Schank, Hayes, expert systems, anthropology.12Problems with the classical theory: Plato’s problem• Reading from Wittgenstein• Fodor’s example of painting.– X covers Y with paint – exploding paint factory– X is an agent and X covers Y with paint – kicking over a bucket of paint– X is an agent and X intentionally covers the surface of Y with paint – when you paint a painting, you are not painting the canvas– X is an agent and X intentionally covers the surface of Y with paint and X’s primary intention is to cover Y with paint –dipping painting brush into paint isn’t painting it.• Definitions are, at the least, quite difficultProblems with classical theory: Categories have structure• Prototype effects. – We’ll discuss this more next week. – some elements seem more typical than others.• Sparrow is a more typical bird than