Unformatted text preview:

Introduction to Neural NetworksU. Minn. Psy 5038Daniel KerstenLecture 1-IntroductionGoalUnderstand the functioning of the brain as a computational device. Tools to explain brain & behavior. Relation to connec-tionist, neuromorphic, computational neuroscience research, and cognitive science.Relation to Cognitive ScienceCognitive Science: The interdisciplinary study of the acquisition, storage, retrieval and utilization of knowledge.Problems: perception, learning, memory, planning, actionOften, we don't know how to solve a problem even in principle. For others, we have solutions, but they don't resemble how a biological system might solve the problem.What kinds of problems can large interconnected systems of model neurons solve? What are the limitations? What are the strengths?How do neural networks relate to the larger field of statistical pattern recognition?Understanding the relation between brain and behavior requires...A multidisciplinary approachMultiple levels of explanation.Multidisciplinary approachThree primary areas or disciplines influence current neural network research:‡Neuroscience, computational neuroscience‡Neuroscience, computational neuroscienceUnderstand the basic building blocks or "hardware" of the nervous systemthese are: nerve cells or neurons, and their connections, the synapsesOur emphasis is on: large scale neural networks. Requires great simplification in the model of the neuron...in order to compute and theorize about what large numbers of them can do.Compare with other areas of Computational Neuroscience that emphasize the biology. Here we emphasize "brain-style""computation. Often wrong in detail, but driven by a curiosity about how the complex processes ofperception, and memory work.What can these large scale neural systems do? That is, what can they compute? And how?‡Computational theory, mathematics, statistical pattern recognitionStatistical inference, engineering (information and communication theory), statistical physics and computer science.Provide the tools and analogs to abstract and formalize for analysis and simulation.One of the characteristics of this course is to try to relate the neural models to statistical methods of inference and regres-sion in order to understand the computational principles and power behind a neural implementation.What should these large scale neural systems compute? What are the ways in which information is represented? How can a system be designed to get from input to output representations?‡Behavioral sciences, psychology, cognitive science and ethologyUnderstand what subsystems are supposed to do as a functioning organism in the environment.Psychology & Computational theory =>The brain is NOT a general purpose computer.2 Lect_1_Introduction.nb‡How do different theoretical neural network approaches relate? A little history and future...Statistical patternrecognition ê machinelearning--behavioral ê functional testsComputationalneuroscience--neural tests1950sPresentBayesianmodels?Multiple levels of explanation‡Functional/Behavioral levelPsychology/Cognitive Science/Ethology tells us what is actually solved by functioning behaving organisms. Descriptions of behavior.Lect_1_Introduction.nb 3‡Statistical Inference levelTheories of pattern recognition, inference & estimation.Functionalities supported by neural network computing provide a useful way of categorizing models in terms of the computational tasks required:1. Learning input/ouput mappings from examples (learning as regression, classification boundaries)associative memory (->neural networks: Hopfield net, back-prop, local minima can be useful.)2. Inferring outputs from inputs (continuous estimation, discrete classification)memory recall, perceptual inferenceoptimization or constraint satisfaction (->neural networks: Hopfield net, Boltzmann machine, global minimum is desired, local minima are problems)3. Modeling data (learning as probability density estimation)self-organization of sensory data into useful representations or classes (e.g principal components analysis, cluster-ing)(Can view 1. Learning input/output mappings as a special case)‡Neural network level: Algorithms, implementationAlgorithms: Mathematics of computation tells us what is computable and how. Practical limits. Parallel vs. serial.Input and output representation, and algorithms for getting from input to output. Programming rules, data structures.Implementation: Wetware, hardware.Neuroscience, neurophysiology and anatomy tell us the adequacies and inadequacies of our modeling assumptions.‡Emphasis in this courseUnderstand high-level functions such as vision, pattern recognition, learning, memory, inference and control. In the brain these functions involve large-scale systems each with many "modules" and 10s to 100s of thousands of neurons in each. Appropriate level tends towards more abstract where we can manage the complexity through mathematical models.The interaction between levels of analysis considers a function (e.g. pattern recognition), the theory to understand the function (e.g. through statistical inference), and how the function may be realized in a neural system (neural networks):4 Lect_1_Introduction.nbThe Big Picture: Overview of the BrainBefore we look at models of neurons and their interactions, let us get an overview of the large scale context. Understanding function means understanding how an organism's information processing is determined by the structure of its environmental inputs (e.g. natural images, objects to be avoided, places to go), and the nature of its outputs (e.g. estimates of shapes of objects to be manipulated, terrain to walk on, movements, and decisions). The brain doesn't operate in isolation, these inputs and outputs are intimately tied to the sensory and motor neurons that make up the peripheral nervous system, as well as the physical make-up of the body itself.The brain has both surface and interior structures. Surface structures that are visible in the side view below are: frontal, temporal, parietal and occipital lobes, and the cerebellum. But apart from the cerebellum, it isn't totally obvious where one part stops and another begins. Landmarks are the sulci (valleys) and gyri (bumps). E.g. the lateral fissure (sulcus) is perhaps the easiest to spot. It separates the temporal lobe from the frontal and parietal lobes.There are internal components too: Thalamus (sensory and motor relays), hypothalamus (control of endocrine


View Full Document
Download Neural Networks
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Neural Networks and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Neural Networks 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?