View Full Document


Unformatted text preview:

Optimal In Place Self Organization for Cortical Development Limited Cells Sparse Coding and Cortical Topography Juyang Weng and Matthew D Luciw Department of Computer Science and Engineering Michigan State University East Lansing MI 48824 USA Abstract Cortical self organization during open ended development is a core issue for perceptual development Traditionally unsupervised learning and supervised learning are two different types of learning conducted by different networks However there is no evidence that the biological nervous system treats them in a disintegrated way The computational model presented here integrates both types of learning using a new biologically inspired network whose learning is in place By in place learning we mean that each neuron in the network learns on its own while interacting with other neurons There is no need for a separate learning network We present in this paper the Multi layer In place Learning Network MILN for regression and classification This work concentrates on its two layer version for global pattern detection without incorporating an attention selection mechanism It reports properties about limited cells sparse coding and cortical topography The network enables both unsupervised and supervised learning to occur concurrently Within each layer the adaptation of each neuron is nearly optimal in the sense of the least possible estimation error given the observations Experimental results are presented to show the effects of the properties investigated Index Terms Biological cortical learning statistical efficiency minimum error self organization incremental learning I I NTRODUCTION What are the possible mechanisms that lead to the emergence of the orientation cells in V1 Since V1 takes input from the retina LGN and other cortical areas the issue points to the developmental mechanisms for the formation and adaptation of the multi layer pathways of visual processing Well known unsupervised learning algorithms include SelfOrganizing Map SOM vector quantization PCA Independent Component Analysis ICA Isomap and Non negative Matrix Factorization NMF Only a few of these algorithms have been expressed by in place versions e g SOM and PCA 11 Supervised learning networks include feed forward networks with back propagation learning radial basis functions with iterative model fitting based on gradient or similar principles Cascade Correlation Learning Architecture 2 support vector machines SVM and Hierarchical Discriminant Regression HDR 3 However it is not convincing that biological networks use two different types of networks for unsupervised and supervised learning which occur in an intertwined way in the process of development When a child learns to draw his parent can hold his hand during some periods to guide his hand movement i e supervised but leave him practicing on his own during other periods i e unsupervised Does the brain switch between two totally different networks one for supervised moments and the other for unsupervised moments The answer to this type of question is not clear at the current stage of knowledge However there is evidence that the cortex has wide spread projections both bottom up and top down 8 pages 99 103 For example cells in layer 6 in V1 project back to the lateral geniculate nucleus 5 page 533 Can projections from later cortical areas be used as supervision signals Currently there is a lack of biologically inspired networks that integrate these two different learning modes using a single learning network The network model proposed here enables unsupervised and supervised learning to take place at the same time throughout the network One of the major advantages of supervised learning is the development of certain invariant representations Some networks have built in programmed in invariance either spatial temporal or some other signal properties Other networks do not have built in invariance The required global invariance then must be learned object by object However they cannot share invariance of subparts or locally invariant features for different objects Consequently the number of samples needed to reach the desired global invariance in object recognition is very large This paper proposes a new general purpose multi layer network which learns invariance from experience The network is biologically inspired The network has multiple layers later layers take the response from early layers as their input This work concentrates on two layers The network enables supervision from two types of projections a supervision from the succeeding layer b supervision from other cortical regions e g as attention selection signals The network is self organized with unsupervised signals input data from bottom up and supervised signals motor signals attention selection etc from top down From a mathematical point of view in each layer of the network unsupervised learning enables nodes neurons to generate a self organized map that approximates the statistical distribution of the bottom up signals input vector space while supervised learning adjusts the node density in such a map so that those areas in the input space that are not related or weakly related to the output from this layer receive no or fewer nodes Therefore more nodes in each layer will respond to output relevant input components This property leads to increasing invariance from one layer to the next in a multi layer network Finally global invariance emerges at the last motor layer Furthermore we require in place learning By in place learning we mean that the signal processing network itself deals with its own adaptation through its own internal physiological mechanisms and interactions with other connected networks and thus there is no need to have an extra network that accomplishes the leaning adaptation It is apparent that a design of an in place learning biologically inspired network that integrates both unsupervised and supervised learning is not trivial In what follows we first present the network structure in Section II Then in Section III we explain the in place learning mechanism within each layer Experimental examples that demonstrate the effects of the discussed principles are presented in Section IV Section V provides some concluding remarks II T HE M ULTI L AYER I N PLACE L EARNING N ETWORK This section presents the architecture of the new Multilayer In place Learning Networks MILN whose architecture is shown in Fig 1 For biological plausibility assume that the

Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...

Join to view LCA-2 and access 3M+ class-specific study document.

We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view LCA-2 and access 3M+ class-specific study document.


By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?