Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 9: Neural spatial filtering‡Initialize:In[68]:=Off@General::spell1D;SetOptions@ArrayPlot, Mesh -> False, AspectRatio -> Automatic,PlotRange -> All, ColorFunction Ø "GrayTones", DataReversed Ø True,ImageSize Ø SmallD;OutlineLast time: Key ideas1. The output or response of a linear system can be modeled as a matrix operation. If the system is shift-invariant, the matrix has the form of a convolution.2. Some representations of images are better than others depending on the job of the model:Spatial frequency representation of images good for modeling optical transformation. E.g. Eigenfunctions of the system: Avoids convolution--one can project the image onto the appropriate basis set providing the spectrum. Then scale each eigenvector by the product of spectrum with the MTF (i.e. eigenvalues of the system), and then add them all up.This timeLinear systems models of neural processingSingle-channel spatial filteringMultiple channel filtersPsychophysical experiments. ->Multi-resolution, and wavelet bases->A model of the spatial filtering properties of neurons in the primary visual cortexUnderstanding the material in this lecture will provide a basis for understanding current research in:The search for the neural basis of image feature extraction for image representation and recognitionComputer vision models for edge detection, texture processing, ...Models of human image discrimination performance, image quality metrics, ...Tutorials‡Mathematica tutorial on convolutions‡Mathematica tutorial on fourier analysis of imagesSingle channel spatial frequency filtering Mach bands & perceptionErnst Mach was a 19th century physicist and philosoper known today for a unit of speed and for "Mach's principle", Mach was also interested in sensory physiology and today is also known for several visual illusions. One illusion is called "Mach bands". He noticed that the brightness of a luminance ramp didn’t look like one would predict simply from physical measurements of light intensity. Let's make some Mach bands. 2 9.NeuralSpatialFiltering.nbIn[88]:=lb = 40;ub= 80;size = 120;Clear[y];low = 0.2; hi = 0.8;y[x_,lb_,ub_] := low /; x<lby[x_,lb_,ub_] :=((hi-low)/(size/3)) x + (low-(hi-low)) /; x>=lb && x<uby[x_,lb_,ub_] := hi /; x>=ubIn[85]:=machg = Plot@y@x, 40, 80D, 8x, 0, 120<, PlotRange Ø 880, 120<, 80, 1<<,PlotStyle Ø [email protected]<DOut[85]=0204060801001200.00.20.40.60.81.0We'll now make a 2D gray-level picture with ListDensityPlot to experience the Mach bands for ourselves. PlotRange allows us to scale the brightness.9.NeuralSpatialFiltering.nb 3In[96]:=picture = TableBTable@y@i, lb, ubD, 8i, 1, size<D, :i, 1,size2>F;ArrayPlot@picture, Frame Ø False, PlotRange Ø 80, 1<DOut[97]=There...what took Mach some effort to set up carefully only requires a computer, some general purpose software, and a few lines of code.What Mach noticed was that the left knee of the ramp looked too dark, and the right knee looked too bright. Objective light intensity did not predict apparent brightness.The red line below is proportional to the actual intensity. The blue line shows an informal sketch of apparent brightness.Why is this? Mach advanced an explanation in terms of lateral inhibition, an explanation that hasn’t changed much in over 100 years. We’ll return to it below. But first let’s take a look at what is known about retinal anatomy and physiology.‡Mach's explanation4 9.NeuralSpatialFiltering.nbPhysiological basis for spatial filteringThe neural basis? Lateral inhibitory filtering in sensory cells may be part of the answer. Found in vertebrates and invertabrates: Limulus (horseshoe crab)--Hartline, who won the 1967 Nobel prize for this work that began in the 30's;vertebrates: Frog - Barlow, and mammals: Cat --Kuffler. Below is a schematic representation of the mammalian retina.The receptors are connected to horizontal and bipolar cells which in turn are connected to amacrine cells. One overall function of this circuitry is to provide gain control for intensity signals, with high-pass spatial and temporal filtering, so that the retinal output signals light levels relative to an average level, rather than signalling absolute level.The function of the retina may best be seen by a close examination of what its output cells are doing. These cells are called ganglion cells and they send their axons out of the eye through the optic disk ("blind spot") to the lateral geniculate nucleus (LGN). The LGN has been crudely (and perhaps erroneously) likened to a relay station en route to cortex. Several types of ganglion cells, each with distinctive anatomy and function have been identified in cat and monkey (Enroth-Cugell and Robson, 1966; see Shapley and Perry, 1986 for a comparison with monkey ganglion cells). In cat, the two principle types are the X, Y cells. They code contrast into trains of action potentials (spikes) whose temporal frequency grows with contrast. In addition, these cells act as approximately circularly symmetric spatial-temporal band-pass filters, with small departures from linearity. What this means should become clear after we explain the idea of a receptive field. If one measures the response of a ganglion cell to a uniformly illuminated screen, one typically finds a mean spike discharge rate (e.g. 50 spikes/sec). Then if a small light spot is positioned on the screen, the cell will increase its firing for some positions, and decrease its firing rate for other positions. A "receptive field" can be mapped out which shows the sensitivity of the cell to the light spots in various locations. One of the striking findings of the 1950's was the discovery that retinal cells almost uniformly show a concentric center-surround organization in which the center is excitatory and the surround inhibitory (or vice versa). The figure below shows a density plot where light areas represent where the cell increases firing to a spot, and the darker areas where it is inhibited9.NeuralSpatialFiltering.nb 5The receptors are connected to horizontal and bipolar cells which in turn are connected to amacrine cells. One overall function of this circuitry is to provide gain control for intensity signals, with high-pass spatial and temporal filtering, so that the retinal output signals light levels relative to an average level, rather than signalling absolute level.The function of the retina may best be seen by a close examination of