DOC PREVIEW
MIT 10 34 - Probability Theory and Stochastic Simulation

This preview shows page 1-2 out of 7 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 7 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 7 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 7 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Ch 7 Probability Theory and Stochastic Simulation: Frequentist statistics: - Probability of observing E: ()NNEpE≈ - Joint Probability: ()()()12121| EEpEpEEp=∩ - Expectation: ∑=≈=exp1exp1)(NWNWWEνν Bayes’ Theorem: ()()()()212121|| EEpEpEEpEp= - Bayes’ Theorem is general. Definitions: - variance: ()()()[]()()[]222var WEWEWEWEW −=−= - (X, Y independent, var(X+Y) = var(X) + var(Y)) - standard deviation: ()Wvar=σ - covariance ()()[]()[]{}YEYXEXEYX−−=,cov , for two random variables X and Y - covariance matrix Important Probability Distributions Definitions: - Discrete random variable o For {}miXXXX ,...,,21= o ()=iXN number of observations of Xi o T is the total number of observations ()TXNMij=∑=1 o Probability is definied by: ()()TXNXPii= o Normalization is defined by ()11=∑=MijXP - Continuous random variable o This is just the continuous version of the above, defined by integrals instead of limits, differentials instead of increments o Normalization condition: ()∫=hixloxdxxp 1 o Expectation () ()∫==hixloxdxxxpxxE - Cumulative Probability distribution o Basis of RAND in matlabo () ()udxxpXFxloxM==∫'' o u is defined as uniformly distributed 10≤≤u Bernoulli trials o Concept that observed error is the net sum of many small random errors Random Walk Problem - key point: independence of coin tosses - Main results: 0=x 22nlx = Binomial Distribution - probability distribution: () ( )()HnnHHnHHHppnnnnP−−= 1, - binomial coefficient: ()!!!HHHnnnnnn−= - BINORND Matlab to generate random number distributed using binomial distribution Gaussian (Normal) Distribution - Take binomial distribution, change into probability of observing net displacement after n steps of length l o ()()()nlxnlxnnlnxp−+=212/!2/!,; - Evaluate in limit that nÆ ∞, take natural log, and use Stirling’s approximation - Algebra, and taylor expand around the ln terms - Taking the exponential and normalizing such that: ()∫∞∞−=1,; dxlnxP - ()−=222exp21;σπσσxxP 22nl=σ - Binomial Distribution of random walk reduces to Gaussian Distribution as nÆ ∞ - Central Limit Theorem: sequence of random variables, which are not distributed normally, the statistic o ∑=−=NjjjjnnS1.1σµξo random variable: jξ with mean .jµ and variance 2jσ - is normally distributed in the limit that nÆ ∞, with variance = 1 o ()−=2exp212nnSSPπ - Non-zero Mean (basis of randn) o ()()−−=2222exp21,σµπσσµxN - Multivariate Gaussian Distribution (use of covariance matrix) o Covariance Matrix: ()[]()[]()[]{}jjiiijEEEννννν−−=cov o Covariance Matrix is always symmetric and positive definite o For independent components: ()I2covσν= o () ()()[]TAxAxA covcovcov ==ν o if ν is a random vector and c is a constant vector: o ()()()[]()[]ccccccTTννννcovcovvarvar ⋅===⋅ o ()()()()−Σ−−Σ=Σ−µνµνπµν12/21exp21,;TNP Poisson Distribution - Poisson distribution can be used to determine probability of success if there are n trials, derived in the limit as nÆ ∞ - Total number of successes in trial is a random variable, which d - Another limiting case of binomial distribution - ()()pnepnpnP−=!,;ξξξ - p = probability of individual success - n = number of trials - ξ = result if success or failure, typically {1,0} with different probabilities Boltzmann/Maxwell Distributions o ()()−=kTqEQqP exp1 o Q is the normalization constant o Replacing E(q) for kinetic energy we arrive at Maxwell Distribution o ()−∝kTmP2exp2νν Brownian Dynamics and Stochastic Differential Equations- velocity autocorrelation function o ()()xVxxtVVeCtCτ/00−≈≥ µρτ922Rxv= o () ( )()tDVtVxxδ20 = - Dirac Delta Function o ()−=→2202exp21limσπσδσtt o ()() ( )0fdtttf =∫∞∞−δ - Langevin equation - Wiener process - Stochastic Differential equations o Explicit Euler SDE method o ()()()()[]()ttxWDtdxdUtxttx ∆+∆−=−∆+2/121ζ - Ito’s Stochastic Calculus o Example: Black-Scholes o Fokker-Planck o Einstein Relation o Brownian motion in multiple dimensions - MCMC o Stat Mech example o Metropolis recipe (pg497) o Example: Ising Lattice o Field theory o Monte Carlo Integration o Simulated annealings o Genetic Programming Bayesian Statistics and Parameter Estimation Goal of this material is to draw conclusions from data (“statistical inference”) and estimate parameters. Basic definitions - Predictor variables: x = [x1 x2 x3… xM] - Response variable: y(R) = [y1 y2 y3 … yL] - θ: model parameters Main goal: match model prediction to that of the observed response by selecting θ. Single-Response Linear Regression- set of predictor variables, known a priori: x[k] = [x1[k] x2[k] x3[k]… xM[k]], for the kth experiment - measurement y[k] - assume a linear model: [][][][][]kkMMkkkxxxyεββββ+++++= ...22110 - the error in ε[k] is responsible for the difference between model and observed - define []TM210 ... ββββθ= - response is: [][]()[]ktruekkxyεθ+⋅= - model prediction is: [][]()truekkxyθ⋅=ˆ - define design matrix X, which contains all information about every experiment (with different predictor variables) −−−−−−−−−−−−−−−−−−=][]2[]1[:NxxxX - vector of predicted responses: ()()()()θθθθθXyyyyN==][]2[]1[:)))) Linear Least Squares Regression - minimize sum of squared errors: ()[] []()[]21∑=−=NkkkyySθθ) - First derivative = 0, 2nd derivative is > 0, using these conditions with above equation you can derive a linear system - ()yXXXTLST=θ Æ ()yXXXTTLS1−=θ (review point?) - XTX, contains information about experimental design to probe the parameter values - XTX is a real, symmetric matrix that is positive-semidefinite - Solving this is through standard linear solving, or QR decomposition or some other method - All this estimates parameters, but does not give us accuracy of our estimates Bayesian view of statistical inference- Statement of belief (especially in random number generators) Bayesian view of


View Full Document

MIT 10 34 - Probability Theory and Stochastic Simulation

Documents in this Course
Load more
Download Probability Theory and Stochastic Simulation
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Probability Theory and Stochastic Simulation and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Probability Theory and Stochastic Simulation 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?