1Stat 35, UCLA, Ivo Dinov Slide 1UCLA STAT 35Applied Computational and InteractiveProbabilityzInstructor: Ivo Dinov, Asst. Prof. In Statistics and NeurologyzTeaching Assistant: Anwar KhanUniversity of California, Los Angeles, Winter 2005http://www.stat.ucla.edu/~dinov/Stat 35, UCLA, Ivo DinovSlide 2Statistics & Their Distributions –The CLTStat 35, UCLA, Ivo DinovSlide 3StatisticA statistic is any quantity whose value can be calculated from sample data. Prior to obtaining data, there is uncertainty as to what value of any particular statistic will result. A statistic is a random variable denoted by an uppercase letter; a lowercase letter is used to represent the calculated or observed value of the statistic.Stat 35, UCLA, Ivo DinovSlide 4Random SamplesThe rv’s X1,…,Xnare said to form a (simplerandom sample of size n if1. The Xi’s are independent rv’s.2. Every Xihas the same probability distribution.Stat 35, UCLA, Ivo DinovSlide 5Simulation ExperimentsThe following characteristics must be specified:1. The statistic of interest.2. The population distribution.3. The sample size n.4. The number of replications k.Stat 35, UCLA, Ivo DinovSlide 6The Distribution of the Sample Mean2Stat 35, UCLA, Ivo DinovSlide 7Using the Sample MeanLet X1,…, Xnbe a random sample from a distribution with mean value and standard deviation Thenµ.σ()()221.2.XXEXVXnµµσσ====In addition, with To = X1+…+ Xn,() ()2, ,and .ooo TETnVTn nµσσ σ== =Stat 35, UCLA, Ivo DinovSlide 8Normal Population DistributionLet X1,…, Xnbe a random sample from a normal distribution with mean value and standard deviation Then for any n, is normally distributed, as is To.µ.σXStat 35, UCLA, Ivo DinovSlide 9The Central Limit TheoremLet X1,…, Xnbe a random sample from a distribution with mean value and variance Then if n sufficiently large, has approximately a normal distribution withXµ2.σ22 and ,XXnσµµ σ==and Toalso hasapproximately a normal distribution with2, .ooTTnnµµσσ==n, the better the approximation.The larger the value ofStat 35, UCLA, Ivo DinovSlide 10The Central Limit TheoremµPopulation distributionsmall to moderate nXlarge nXStat 35, UCLA, Ivo DinovSlide 11Rule of ThumbIf n > 30, the Central Limit Theorem can be used.Stat 35, UCLA, Ivo DinovSlide 12Approximate Lognormal DistributionLet X1,…, Xnbe a random sample from a distribution for which only positive values are possible [P(Xi> 0) = 1]. Then if n is sufficiently large, the product Y = X1X2…Xnhas approximately a lognormal distribution.3Stat 35, UCLA, Ivo DinovSlide 13Central Limit Theorem:When sampling from almost any distribution,is approximately Normally distributed in large samples.X Central Limit Theorem – heuristic formulationShow Sampling Distribution Simulation Applet:file:///C:/Ivo.dir/UCLA_Classes/Winter2002/AdditionalInstructorAids/SamplingDistributionApplet.htmlStat 35, UCLA, Ivo DinovSlide 14Independencez For discrete random variables X and Y, if any one of the following properties is true, the others are also true, and X and Y are independent.(1) fXY(x,y) = fX(x) fY(y) for all x and y(2) fY|x(y) = fY(y) for all x and y with fX(x) > 0(3) fX|y(y) = fX(x) for all x and y with fY(y) > 0(4) P(X ∈ A, Y ∈ B) = P(X ∈ A)P(Y ∈ B) for any sets A and B in the range of X and Y respectively.Stat 35, UCLA, Ivo DinovSlide 15 Stat 35, UCLA, Ivo DinovSlide 16z For the sample mean calculated from a random sample, E( ) = µ and SD( ) = , provided = (X1+X2+ … + Xn)/n, and Xk~N(µ, σ). Thenz ~ N(µ, ). And variability from sample to sample in the sample-means is given by the variability of the individual observations divided by the square root of the sample-size. In a way, averaging decreases variability.X nσRecall we looked at the sampling distribution ofnσX X X X Stat 35, UCLA, Ivo DinovSlide 17Central Limit Effect –Histograms of sample means0.0 0.2 0.40.6 0.81.00120.0 0.2 0.4 0.6 0.8 1.0012n = 10.0 0.2 0.4 0.6 0.8 1.00123n = 2TriangularDistributionSample means from sample sizen=1, n=2, 500 samplesArea = 1210210210Y=2 XStat 35, UCLA, Ivo DinovSlide 18Central Limit Effect -- Histograms of sample means0.0 0.2 0.4 0.6 0.8 1.0n =40.00.2 0.4 0.6 0.8 1.0n = 10Triangular DistributionSample sizes n=4, n=104Stat 35, UCLA, Ivo DinovSlide 19Central Limit Effect –Histograms of sample means0.0 0.2 0.4 0.6 0.8 1.0012n = 10.0 0.20.40.6 0.8 1.0012n = 200.0 0.2 0.4 0.6 0.8 1.0123Area = 1Uniform DistributionSample means from sample sizen=1, n=2, 500 samplesY = XStat 35, UCLA, Ivo DinovSlide 20Central Limit Effect -- Histograms of sample meansn = 40.0 0.2 0.4 0.6 0.8 1.00123n = 100.0 0.2 0.4 0.6 0.81.001234Uniform DistributionSample sizes n=4, n=10Stat 35, UCLA, Ivo DinovSlide 21Central Limit Effect –Histograms of sample meansSample means from sample sizen=1, n=2, 500 samples01234560.00.20.40.60.81.0n = 101234560.00.20.40.60.81.0n = 2012340.00.20.40.60.8Area = 1Exponential Distribution),0[ , ∞∈−xxeStat 35, UCLA, Ivo DinovSlide 22Central Limit Effect -- Histograms of sample meansn = 401230.00.20.40.60.81.0n = 100120.00.40.81.2Exponential DistributionSample sizes n=4, n=10Stat 35, UCLA, Ivo DinovSlide 23Central Limit Effect –Histograms of sample meansSample means from sample sizen=1, n=2, 500 samples0.00.20.4 0.60.8 1.00123n = 10.0 0.2 0.4 0.6 0.8 1.00123n = 20.0 0.2 0.4 0.6 0.8 1.00123Quadratic U DistributionArea = 1()]1,0[ , 12221∈−= xxYStat 35, UCLA, Ivo DinovSlide 24Central Limit Effect -- Histograms of sample meansn=40.0 0.2 0.4 0.6 0.8 1.00123n = 100.0 0.2 0.4 0.6 0.8 1.00123Quadratic U DistributionSample sizes n=4, n=105Stat 35, UCLA, Ivo DinovSlide 25Central Limit Theorem:When sampling from almost any distribution,is approximately Normally distributed in large samples.X Central Limit Theorem – heuristic formulationShow Sampling Distribution Simulation Applet:file:///C:/Ivo.dir/UCLA_Classes/Winter2002/AdditionalInstructorAids/SamplingDistributionApplet.htmlStat 35, UCLA, Ivo DinovSlide 26Let be a sequence of independentobservations from one specific random process. Let and and and both be finite ( ). If , sample-avg,Then has a distribution which approaches N(µ, σ2/n), as .Central Limit Theorem –theoretical formulation{},...,...,X,XXk21µ=)(XEσ=)(XSD∞<∞<< || ;0µσ∑==nkkXnnX11X∞→nStat 35, UCLA, Ivo DinovSlide