Vision and Visual NeuroscienceTomaso PoggioJim Mutch + Hueihan JhuangClass 14-15Wednesday, March 31, 2010 Class 14: HLM in the ventral stream of visual cortex Class 15 Models of the ventral an dorsal stream Class 16: Derived Kernels: a mathematical framework for hierarchical learning machines  Class 17: Attention: a Bayesian extension of the modelPlan for class 14-15-16-17 Wednesday, March 31, 2010How then do the learning machines described in the theory compare with brains?  One of the most obvious differences is the ability of people and animals to learn from very few examples. The algorithms we have described can learn an object recognition task from a few thousand labeled images but a child, or even a monkey, can learn the same task from just a few examples. Thus an important area for future theoretical and experimental work is learning from partially labeled examples  A comparison with real brains offers another, related, challenge to learning theory. The “learning algorithms” we have described in this paper correspond to one-layer architectures. Are hierarchical architectures with more layers justifiable in terms of learning theory? It seems that the learning theory of the type we have outlined does not offer any general argument in favor of hierarchical learning machines for regression or classification.  Why hierarchies? There may be reasons of efficiency – computational speed and use of computational resources. For instance, the lowest levels of the hierarchy may represent a dictionary of features that can be shared across multiple classification tasks. There may also be the more fundamental issue of sample complexity. Learning theory shows that the difficulty of a learning task depends on the size of the required hypothesis space. This complexity determines in turn how many training examples are needed to achieve a given level of generalization error. Thus our ability of learning from just a few examples, and its limitations, may be related to the hierarchical architecture of cortex. Notices of the American Mathematical Society (AMS), Vol. 50, No. 5,537-544, 2003.The Mathematics of Learning: Dealing with DataTomaso Poggio and Steve SmaleWednesday, March 31, 2010Classical learning theory and Kernel Machines (Regularization in RKHS)impliesRemark:Kernel machines correspond toshallow networksX1fXlWednesday, March 31, 2010Winning against the curse of dimensionality: new research directions in learningMany processes - physical processes as well as human activities – generate high-dimensional data: curse of dimensionality or poverty of stimulus. There are, however, basic properties of the data generating process that may allow to circumvent the problem of high dimensionality and make the analysis possible:• smoothness - exploited by L2 regularization techniques• sparsity - exploited by L1 regularization techniques• data geometry - exploited by manifold learning techniques• hierarchical organization – suggested by the architecture of sensory cortexWednesday, March 31, 2010New Research DirectionsWednesday, March 31, 2010This class: using a class of models to summarize/interpret experimental results…with caveats:• Models are cartoons of reality, eg Bohr’s model of the hydrogen atom• All models are “wrong”• Some models can be useful summaries of data and some can be a good starting point for more complete theoriesWednesday, March 31, 20101. Problem of visual recognition, visual cortex2. Historical background3. Neurons and areas in the visual system4. Feedforward hierarchical models• Ventral stream model in more details (Jim Mutch)• Dorsal stream model (Hueihan Jhuang)Wednesday, March 31, 2010unconstrained visual recognition is a difficult learning problem (e.g., “is there an animal in the image?”)The Ventral Stream Wednesday, March 31, 2010Object Recognition and the Ventral Stream Desimone & Ungerleider 1989dorsal stream:“where”ventral stream:“what”Hypothesis: the hierarchy architecture of the ventral stream in monkey visual cortex has a key role in object recognition…of course subcortical pathways may also be important (thalamus, in particular pulvinar…).Wednesday, March 31, 2010Riesenhuber & Poggio 1999, 2000; Serre Kouh Cadieu Knoblich Kreiman & Poggio 2005; Serre Oliva Poggio 2007*Modified from (Gross, 1998) A model of the ventral stream, which is also a hierarchical algorithm… [software available online]Wednesday, March 31, 20101. Problem of visual recognition, visual cortex2. Historical background3. Neurons and areas in the visual system4. Feedforward hierarchical models• Ventral stream model in more details (Jim Mutch)• Dorsal stream model (Hueihan Jhuang)Wednesday, March 31, 2010Some personal history: First step in developing a model: learning to recognize 3D objects in IT cortexPoggio & Edelman 1990Examples of Visual StimuliWednesday, March 31, 2010An idea for a module for view-invariant identificationArchitecture that accounts for invariances to 3D effects (>1 view needed to learn!)Regularization Network (GRBF)with Gaussian kernelsView AngleVIEW-INVARIANT, OBJECT-SPECIFICUNITPrediction: neurons becomeview-tuned through learningPoggio & Edelman 1990Wednesday, March 31, 2010Learning to Recognize 3D Objects in IT CortexLogothetis Pauls & Poggio 1995Examples of Visual StimuliAfter human psychophysics (Buelthoff, Edelman, Tarr, Sinha, to be added next year…), which supports models based on view-tuned units... … physiology!Wednesday, March 31, 2010Recording Sites in Anterior ITLogothetis, Pauls & Poggio 1995…neurons tuned to faces are intermingled nearby….Wednesday, March 31, 2010Neurons tuned to object views, as predicted by model!Logothetis Pauls & Poggio 1995Wednesday, March 31, 2010A “View-Tuned” IT Cell12 7224 8448 10860120369612 24 36 48 60 72 84 96 108 120 132 168o o o o o o o o o oo o-108 -96-84 -72-60-48 -36 -24 -12 0-168 -120DistractorsTarget Views60 spikes/sec800 msec-108 -96 -84 -72 -60 -48 -36 -24 -12 0-168 -120oo o o ooo o o oo oLogothetis Pauls & Poggio 1995Wednesday, March 31, 2010But also view-invariant object-specific neurons (5 of them over 1000 recordings)Logothetis Pauls & Poggio 1995Wednesday, March 31, 2010View-tuned cells: scale invariance (one training view only) motivates present modelLogothetis Pauls & Poggio 1995Wednesday, March 31, 2010Hierarchy• Gaussian centers (Gaussian Kernels) tuned to complex multidimensional features as