# U of M PSY 5038 - Lecture notes (33 pages)

Previewing pages*1, 2, 15, 16, 17, 32, 33*of 33 page document

**View the full content.**## Lecture notes

Previewing pages
*1, 2, 15, 16, 17, 32, 33*
of
actual document.

**View the full content.**View Full Document

## Lecture notes

0 0 90 views

- Pages:
- 33
- School:
- University of Minnesota- Twin Cities
- Course:
- Psy 5038 - Introduction to Neural Networks

**Unformatted text preview:**

Introduction to Neural Networks U Minn Psy 5038 Gaussian generative models learning and inference Initialize standard library files Off General spell1D Last time Quick review of probability and statistics Applications to random sampling If we know p x and are given a function y f x what is p y pY HyL dy pX HxL dx This principle is used to make random number generators for general probability densities from the uniform distribution The result is that one can make a random draw from a uniform distribution p x from between 0 and 1 and go to the inverse CDF to read off the value of the random sample from p y 2 Lect 23 GaussGen nb Today Review of big picture Examples of computations on continuous probabilities Examples of computations on discrete probabilities Introduction to Bayes learning Recall relationship between energy neural networks and Bayesian inference Here we are only talking about inference or estimation based on patterns of neural activity i e in the language of neural networks about recall rather than learning Later we will introduce Bayesian learning In the general case we can talk about the probability over all possible values of a neural network s state vector p HV1 V2 L This doesn t distinguish which values are fixed and which are allowed to vary If some values are fixed then we can treat those as the input and allow the network s free neurons to vary to maximize a conditional probability Relationship between posterior probability and the energy of the state of a Hopfield neural network H the hypothesis space corresponds to the values of state variables i e patterns of neural activity that are changing V1 V2 d corresponds to data or the fixed clamped values V1s V2s From the more abstract point of view of statistical inference we have some variables that are fixed the data d some we let vary to maximize probability the hypotheses H and some that we may not care to estimate n Let s see how to use these distinctions Suppose we have p H d n Two rules of inference Marginalize over what you don t care about Lect 23 GaussGen nb pHH dL pHH d nL n Condition on what you know pHH dL pHH dL pHdL Neural population codes and probability distributions Energy methods show how a population of neurons could interact to compute a single value corresponding to the most probable solution Recent behavioral studies show that humans make decisions that combine information so as to take into account uncertainty How might neural populations represent uncertainty Probability distributions See Pouget et al 2006 in the Readings Recall the ideas of population codes from Lecture 16 where a stimulus attribute might be the orientation of a line and the activity or spike count ri the response of the ith unit Each unit i has a tuning function with a preferred orientation In lecture 16 we showed how the notion of a population vector has been applied to explaining a diverse range of phenomena including adaptation effects and motor planning through the computation of a single estimate analogous to the center of mass But do population codes represent just single values E g suppose vector r repesents the pattern of spike counts over a population of orientation tuned neurons We saw how to estimate the value of orientation s which that population represents 3 4 Lect 23 GaussGen nb But what if more information could be represented and used that includes knowledge of the uncertainty or more generally the posterior distribution of s given r p s r Poisson model p r s is a reasonable first approximation to the variability that results in spike counts for repeated applications of the same stimulus To compute with distributions requires a mechanism that can combine information from more than one distribution Pouget and colleagues have shown that poisson like distributions have a special status in that p s r1 p s r2 is proportional to p s r1 r2 Figures adapted from lecture by Alex Pouget Mathematica functions for gaussian multivariates exploring marginals Lect 23 GaussGen nb 5 Mathematica functions for gaussian multivariates exploring marginals Define PDF CDF MultinormalDistributionAm SE specifies a multinormal Hmultivariate GaussianL distribution with mean vector m and covariance matrix In 253 m1 1 1 2 r 1 2 1 2 3 2 3 4 ndist MultinormalDistribution m1 r pdf PDF ndist x1 x2 1 3 2 K H 1 x1L K 9 3 1 3 9 1 1 H 1 x1L K x2OO K H 1 x1L K x2OO K x2OO 4 8 2 8 16 2 2 Out 256 4 In 257 Out 257 2 p g1 ContourPlot PDF ndist 8x1 x2 D 8x1 3 3 8x2 3 3 D 6 Lect 23 GaussGen nb What is the probability of the distribution in the region x1 5 x2 2 In 240 grp RegionPlot x1 5 x2 2 8x1 4 4 8x2 4 4 PlotStyle Directive Opacity 25D EdgeForm D FaceForm GrayDDD Show 8 g1 grp ImageSize SmallD Out 241 In 242 gcdf ContourPlot CDF ndist x1 x2 x1 4 4 x2 4 4 ImageSize Small Show gcdf grp ImageSize Small Out 243 CDF ndist 8 5 2 0 D 0 225562 Lect 23 GaussGen nb 7 Calculating the marginals In 244 Clear x1 x2D marginal x1 D PDF ndist 8x1 x2 D x2 marginal2 x2 D PDF ndist 8x1 x2 D x1 In 247 mt Table 8x1 marginal x1D 8x1 3 3 2 D g2 ListPlot mt Joined True PlotStyle 8Red Thick Axes FalseD In 249 mt2 Table 8x2 marginal2 x2D 8x2 3 3 4 D g3 ListPlot mt2 Joined True PlotStyle 8Green Thick Axes FalseD In 251 theta Pi 2 Show g1 Epilog 8Inset g2 80 3 80 0 D Inset g3 8 3 0 80 0 Automatic 88Cos thetaD Sin thetaD 8Sin thetaD Cos thetaD D D Out 252 Finding the mode For the Gaussian case the mode vector corresponds to the mean vector But we can pretend we don t know that and find the maximum and the coordinates where the max occurs 8 Lect 23 GaussGen nb In 260 FindMaximum PDF ndist 8x1 x2 D 88x1 0 8x2 0 D Out 260 80 168809 8x1 1 x2 0 5 Drawing samples As we ve used in earlier lectures drawing samples is done by RandomReal ndistD 81 07766 0 280209 Mixtures of gaussians with MultinormalDistribution Multivariate gaussian distributions are often inadequate to model real life problems that for example might involve more than one mode One solution is to approximate more general distributions by a sum or mixture of gaussians In 261 Clear mixD In 262 r1 0 4 1 6 6 1 r2 0 4 1 6 6 1 m1 1 5 m2 1 5 ndist1 MultinormalDistribution m1 r1 ndist2 MultinormalDistribution m2 r2 In 267 mix x 0 5 PDF ndist1 x PDF ndist2 x Lect 23 GaussGen nb In 268 9 gg1 ContourPlot mix 8x1 x2 D 8x1 2 2 8x2 2 2 PlotRange Full ImageSize SmallD Out 268 Marginals for mixture marginal x1 D Integrate mix 8x1 x2 D 8x2 Infinity Infinity D In 269 Clear …

View Full Document