CS 561, Session 281Artificial Neural Networks and AIArtificial Neural Networks provide…- A new computing paradigm- A technique for developing trainable classifiers, memories, dimension-reducing mappings, etc- A tool to study brain functionCS 561, Session 282Converging Frameworks• Artificial intelligence (AI): build a “packet of intelligence” into a machine• Cognitive psychology: explain human behavior by interacting processes (schemas) “in the head” but not localized in the brain• Brain Theory: interactions of components of the brain -- computational neuroscience - neurologically constrained-models• and abstracting from them as both Artificial intelligence andCognitive psychology:- connectionism: networks of trainable “quasi-neurons” to provide “parallel distributed models” little constrained by neurophysiology- abstract (computer program or control system) information processing modelsCS 561, Session 283Vision, AI and ANNs• 1940s: beginning of Artificial Neural NetworksMcCullogh & Pitts, 1942Σi wixi ≥θPerceptron learning rule (Rosenblatt, 1962)BackpropagationHopfield networks (1982)Kohonen self-organizing maps… input outputneuronmMSmCS 561, Session 284Vision, AI and ANNs1950s: beginning of computer visionAim: give to machines same or better vision capability as oursDrive: AI, robotics applications and factory automationInitially: passive, feedforward, layered and hierarchical processthat was just going to provide input to higher reasoningprocesses (from AI)But soon: realized that could not handle real images1980s: Active vision: make the system more robust by allowing thevision to adapt with the ongoing recognition/interpretationCS 561, Session 285CS 561, Session 286CS 561, Session 287Major Functional Areas• Primary motor: voluntary movement• Primary somatosensory: tactile, pain, pressure, position, temp., mvt.• Motor association: coordination of complex movements• Sensory association: processing of multisensorial information• Prefrontal: planning, emotion, judgement• Speech center (Broca’s area): speech production and articulation• Wernicke’s area: comprehen-• sion of speech• Auditory: hearing• Auditory association: complex• auditory processing• Visual: low-level vision• Visual association: higher-level• visionCS 561, Session 288Felleman & Van Essen, 1991InterconnectCS 561, Session 289More on ConnectivityCS 561, Session 2810Neurons and SynapsesCS 561, Session 2811Electron Micrograph of a Real NeuronCS 561, Session 2812Transmenbrane Ionic Transport•Ion channelsact as gates that allow or block the flow of specific ions into and out of the cell.CS 561, Session 2813The Cable Equation• See http://diwww.epfl.ch/~gerstner/SPNM/SPNM.htmlfor excellent additional material (some reproduced here).• Just a piece of passive dendrite can yield complicated differential equations which have been extensively studied by electronicians in the context of the study of coaxial cables (TV antenna cable):CS 561, Session 2814The Hodgkin-Huxley ModelExample spike trains obtained…CS 561, Session 2815Detailed Neural Modeling• A simulator, called “Neuron” has been developedat Yale to simulate the Hodgkin-Huxley equations,as well as other membranes/channels/etc.See http://www.neuron.yale.edu/CS 561, Session 2816The "basic" biological neuron• The soma and dendrites act as the input surface; the axon carries the outputs. • The tips of the branches of the axon form synapses upon other neurons or upon effectors (though synapses may occur along the branches of an axon as well as the ends). The arrows indicate the direction of "typical" information flow from inputs to outputs.Dendrites Soma Axon with branches andsynaptic terminalsCS 561, Session 2817• A McCulloch-Pitts neuron operates on a discrete time-scale, t = 0,1,2,3, ... with time tick equal to one refractory period• At each time step, an input or output is onoroff — 1 or 0, respectively. • Each connection or synapse from the output of one neuron to the input of another, has an attached weight. Warren McCulloch and Walter Pitts (1943)x (t)1x (t)nx (t)2y(t+1)w12nwwaxonθθθθCS 561, Session 2818Excitatory and Inhibitory Synapses• We call a synapseexcitatory if wi> 0, andinhibitory if wi< 0. • We also associate a thresholdθ with each neuron• A neuron fires (i.e., has value 1 on its output line) at time t+1 if the weighted sum of inputs at t reaches or passes θ:y(t+1) = 1 if and only if ΣΣΣΣ wixi(t) ≥≥≥≥ θθθθCS 561, Session 2819From Logical Neurons to Finite AutomataAND111.5NOT-10OR110.5Brains, Machines, and Mathematics, 2nd Edition, 1987X Y→Boolean NetXYQFinite AutomatonCS 561, Session 2820Increasing the Realism of Neuron Models• The McCulloch-Pitts neuron of 1943 is importantas a basis for • logical analysis of the neurally computable, and• current design of some neural devices (especially when augmented by learning rules to adjust synaptic weights). • However, it is no longer considered a useful model for making contact with neurophysiological data concerning real neurons.CS 561, Session 2821Leaky Integrator Neuron• The simplest "realistic" neuron model is a continuous time model based on using the firing rate (e.g., the number of spikes traversing the axon in the most recent 20 msec.) as a continuously varying measure of the cell's activity• The state of the neuron is described by a single variable, the membrane potential. • The firing rate is approximated by a sigmoid, function of membrane potential.CS 561, Session 2822Leaky Integrator Modelτ = - m(t) + h has solution m(t) = e-t/ττττm(0) + (1 - e-t/ττττ)h → h for time constant τ > 0. • We now add synaptic inputs to get the Leaky Integrator Model:τ = - m(t) + ΣiwiXi(t) + hwhere Xi(t) is the firing rate at the ithinput. • Excitatory input (wi> 0) will increase • Inhibitory input (wi< 0) will have the opposite effect.m(t)m(t)m(t)CS 561, Session 2823Hopfield Networks• A paper by John Hopfield in 1982 was the catalyst in attracting the attention of many physicists to "Neural Networks".• In a network of McCulloch-Pitts neuronswhose output is 1 iff Σwij sj ≥θi and is otherwise 0,neurons are updated synchronously: every neuron processes its inputs at each time step to determine a new output.CS 561, Session 2824Hopfield Networks• A Hopfield net (Hopfield
View Full Document
Unlocking...