Learning to See Rotation andDilation with a Hebb RuleMartin I. Sereno and Margaret E. SerenoCognitive Science D-015University of California, San DiegoLa Jolla, CA 92093-0115AbstractPrevious work (M.I. Sereno, 1989; cf. M.E. Sereno, 1987) showed that afeedforward network with area V1-like input-layer units and a Hebb rulecan develop area MT-like second layer units that solve the apertureproblem for pattern motion. The present study extends this earlier workto more complex motions. Saito et al. (1986) showed that neurons withlarge receptive fields in macaque visual area MST are sensitive todifferent senses of rotation and dilation, irrespective of the receptive fieldlocation of the movement singularity. A network with an MT-likesecond layer was trained and tested on combinations of rotating, dilating,and translating patterns. Third-layer units learn to detect specific sensesof rotation or dilation in a position-independent fashion, despite havingposition-dependent direction selectivity within their receptive fields.1 INTRODUCTIONThe visual systems of mammals and especially primates are capable of prodigious feats ofmovement, object, and scene recognition under noisy conditions--feats we would like tocopy with artificial networks. We are just beginning to understand how biologicalnetworks are wired up during development and during learning in the adult. Even at thisstage, however, it is clear that explicit error signals and the apparatus for propagating thembackwards across layers are probably not involved. On the other hand, there is a growingbody of evidence for connections whose strength can be modified (via NMDA channels)as functions of the correlation between pre- and post-synaptic activity. The present projectwas to try to learn to detect pattern rotation and dilation by example, using a simple HebbIn: R.P. Lippmann, J. Moody, and D.S. Touretzky, eds. (1991), Advances in Neural Information Processing Systems 3. San Mateo, CA: Morgan Kaufmann Publishers, pp. 320-326.rule. By building up complex filters in stages using a simple, realistic learning rule, wereduce the complexity of what must be learned with more explicit supervision at higherlevels.1.1 ORIENTATION SELECTIVITYSome of the connections responsible for the selectivity of cortical neurons to local stimulusfeatures develop in the absence of patterned visual experience. For example, primaryvisual cortex (V1 or area 17) contains orientation-selective neurons at birth in severalanimals. Linsker (1986a,b) has shown that feedforward networks with gaussiantopographic interlayer connections, linear summation, and simple hebb rules, developorientation selective units in higher layers when trained on noise. In his linear system,weight updates for a layer can be written as a function of the two-point correlationcharacterizing the previous layer. Noise applied to the input layer causes the emergence ofconnections that generate gaussian correlations at the second layer. This in turn drives thedevelopment of more complex correlation functions in the third layer (e.g., difference-of-gaussians). Rotational symmetry is broken in higher layers with the emergence of Gabor-function-like connection patterns reminiscent of simple cells in the cortex.1.2 PATTERN MOTION SELECTIVITYThe ability to see coherent motion fields develops late in primates. Human babies, forexample, fail to see the transition from unstructured to structured motion--e.g., thetransition between randomly moving dots and circular 2-D motion--for several months.The transition from horizontally moving dots with random y-axis velocities to dots withsinusoidal y-axis velocities (which gives the percept of a rotating 3-D cylinder) is seen evenlater (Spitz, Stiles-Davis, & Siegel, 1988). This suggests that the cortex requires manyexperiences of moving displays in order to learn how to recognize the various types ofcoherent texture motions.However, orientation gradients, shape from shading, and pattern translation, dilation, androtation cannot be detected with the kinds of filters that can be generated solely by noise.The correlations present in visual scenes are required in order for these higher level filtersto arise.1.3 NEUROPHYSIOLOGICAL MOTIVATIONMoving stimuli are processed in successive stages in primate visual cortical areas. The firstcortical stage is layer 4Cα of V1, which receives its main ascending input from themagnocellular layers of the lateral geniculate nucleus. Layer 4Cα projects to layer 4B,which contains many tightly-tuned direction-selective neurons. These neurons, however,respond to moving contours as if these contours were moving perpendicular to their localorientation (Movshon et al., 1985).Layer 4B neurons project directly and indirectly to area MT, where a subset of neuronsshow a relatively narrow peak in the direction tuning curve for a plaid that is lined up withthe peak for a single grating. These neurons therefore solve the aperture problem forpattern translation presented to them by the local motion detectors in layer 4B of V1. MTneurons, however, appear to be largely blind to the sense of pattern rotation or dilation(Saito et al., 1986). Thus, there is a higher order ’aperture problem’ that is solved by theneurons in the parts of areas MST and 7a that distinguish senses of pattern rotation anddilation. The present model provides a rationale for how these stages might naturally arisein development.2 RESULTSIn previous work (M.I. Sereno, 1989; cf. M.E. Sereno, 1987) a simple 2-layer feedforwardarchitecture sufficed for an MT-like solution to the aperture problem for local translationalmotion. Units in the first layer were granted tuning curves like those in V1, layer 4B. Eachfirst-layer unit responded to a particular range of directions and speeds of the componentof movement perpendicular to a local contour. Second layer units developed MT-likereceptive fields that solved the aperture problem for local pattern translation when trainedon locally jiggled gratings rigidly moving in randomly chosen pattern directions.2.1 NETWORK ARCHITECTUREA similar architecture was used for second-to-third layer connections (see Fig. 1--a samplenetwork with 5 directions and 3 speeds). As with Linsker, a new input layer wasconstructed from a canonical unit, suitably transformed. Thus,