Appendix A4: Normal DistributionBasic Definitions and Facts from Statistics• A random variable can be viewed as the name of an experiment with a probabilistic outcome. Its value is the outcome of the experiment.• A probability distribution for a random variable Y specifies the probability Pr(Y=yi) that Y will take on the value of yifor each possible value of yi. • The expected value, or mean, of a random variable Y is . • The symbol is commonly used to represent E[Y].• The standard deviation of Y is . • The symbol is often used to represent the standard deviation of Y.)Pr(][∑==iiiyYyYE)(YVarYµYσMean Expected Value (also called mean value) is the average of the values taken on by repeatedly sampling the random variableDefinition: Consider a random variable Y that takes on the possible values y1,…yn. The expected value of Y, E[Y], is If Y takes value 1 with prob .7 and value 2 with prob .3 then expected value is 1x0.7+2x0.3=1.3)Pr(][1iniiyYyYE =≡∑=Mean and VarianceVariance captures how far the random variable is expected to vary from its mean valueDefinition: The variance of a random variable Y, Var[Y], isIf Y is governed by a Binomial DistributionE[Y] = npe.g., if n =50 and p = .2 then E[Y]= 25]])[[(][2YEYEYVar −≡Standard DeviationThe square root of the variance is called the standard deviationDefinition: The standard deviation of a random variable Y is]2])[[( YEYEY−≡σBasic Definitions and Facts• The Binomial distribution gives the probability of observing r heads in a series of n independent coin tosses, if the probability of heads in a single toss is p.• The Normal distribution is a bell-shaped probability distribution that covers many natural phenomena.• The Central Limit Theorem states that the sum of a large number of independent, identically distributed random variables approximately follows a Normal distribution.The Binomial DistributionProbability of observing r headsfrom n coin flipexperimentsForm of Binomialdistribution dependson sample size nand probability por errorD(h)Basic Definitions and Facts, continued• An estimator is a random variable Y used to estimate some parameter p of an underlying population.• The estimation bias of Y as an estimator for p is the quantity (E[Y]-p). An unbiased estimator is one for which the bias is zero.• An N% confidence interval estimate for parameter p is an interval that includes p with probability N%.Table 5.2Estimators, Bias and Variance Definition: The estimation bias of an estimator Y for an arbitrary parameter p isIf the estimation bias is zero, then Y is an unbiased estimator for perrorS(h) is an unbiased estimate of errorD(h), since expected value of r is np, and since n is constant expected value of r/n is ppYE−][The Normal DistributionTable 5.4Normal or Gaussian Distribution• Well studied• Tables specify the size of the interval about the mean that contains n% of the probability mass under the normal distributionThe Normal DistributionA bell-shaped distribution defined by the probability density functionIf the random variable x follows a normal distribution, then• The probability that X will fall into the interval (a,b) is given by• The expected, or mean, value of X, E[X], is• The variance of X, Var(X) is• The standard deviation of X, , is 2)(21221)(σµπσ−−=xexp∫badxxp )(µ=][XE2)(σ=xVar2σσσ=xNormal Distribution, Mean 0, Standard Deviation 1With 80% confidence the r.v. will lie in the two-sided interval[-1.28,1.28]Parameters of Normal Density Satisfy:Normally distributed samples tend to cluster around the meanStandardized Random VariableMahanalobis distance, also z-score in one-dimensionProbability is 0.95 that Mahanalobis distance from x to mu is les than 2Standardized r.v. has zero mean unit
View Full Document