Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Lecture 11: Random Samples, Weak Law of Large Numbersand Central Limit TheoremDevore: Section 5.3-5.5October, 2011Page 1Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Definition of a Statistic• A statistic is any quantity whose value can be calculated fromsample data. Prior to obtaining data, there is uncertainty as towhat value of any particular statistic will result.• A statistic is a random variable denoted by an uppercase letter;a lowercase letter is used to represent the calculated orobserved value of the statistic.October, 2011Page 2Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011• Example Consider a sample of n = 3 cars of a particular type;their fuel efficiencies may be x1= 30.7 mpg, x2= 29.4 mpg,x3= 31.1 mpg.• It may also be x1= 28.8 mpg, x2= 30.0 mpg and x3= 31.1mpg• This implies that the value of the mean¯X is different in thesecases. Clearly,¯X is a statistic. The first sample has the mean¯X1= 30.4 mpg and the second one has¯X2≈ 30 mpgOctober, 2011Page 3Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Statistic Examples• A sample mean¯X of the sample X1, . . . , Xnis a statistic; ¯x isone of its possible values• The value of the sample mean from any particular sample canbe regarded as a point estimate of the population µ.• Another example is the sample standard deviation S, while s isits computed value• Yet another example is the difference between the samplemeans for two different populations¯X −¯YOctober, 2011Page 4Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Sampling distribution• Each statistic is a random variable and, as such, has its owndistribution• Consider two samples of size n = 2; if X1= X2= 0,¯X = 0with probability P (X1= 0 ∩ X2= 0)• On the other hand, if X1= 1 but X2= 0 or X1= 0 andX2= 1, we have¯X = 0.5 with probabilityP (X1= 1 ∩ X2= 0) + P (X1= 0 ∩ X2= 1)• This distribution is called the sampling distribution to emphasizeits description of how the statistic varies in value across allpossible sampleOctober, 2011Page 5Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Random Sample• The probability distribution of any statistic depends on thesampling method.• Consider selecting a sample of size n = 2 from the population1, 5, 10. If the sampling is with replacement, it is possible thatX1= X2; then the sampling variance S2= 0 with a nonzeroprobability• However, the sampling without replacement cannot produceS2= 0 and, therefore, P (S2= 0) = 0.October, 2011Page 6Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011(Simple) Random Sample• The RVs X1, . . . , Xnare said to form a simple random sampleof size n if– The Xis are independent RVs.– Every Xihas the same probability distribution.• The usual way to describe these two conditions is to say thatXi’s are independent and identically distributed or iid.October, 2011Page 7Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Simulation Experiments• This is usually employed when the direct derivation is toodifficult• The following characteristics must be specified1. The statistic of interest.2. The population distribution.3. The sample size n.4. The number of replications k.October, 2011Page 8Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Example• Consider the platelet volume distribution in individuals with noknown heart problems. It is commonly assumed to be normal;particular research publication assumes µ = 0.25 andσ = 0.75.• Four experiments are performed, 500 replications each• In the first experiment, 500 samples of n = 5 observationswere generated; in the other three sample sizes were n = 10,n = 20 and n = 30, respectivelyOctober, 2011Page 9Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Distribution of sample mean• Let X1, . . . Xnbe a random sample from a distribution withmean value µ and standard deviation σ. Then1. E(¯X) = µ¯X= µ2. V (¯X) = σ2¯X=σ2n3. σ¯X=σ√n• In addition to the above, for the sample totalT = X1+ X2+ . . . + Xnwe have E T = nµ,V (T ) = nσ2and σT=√nσOctober, 2011Page 10Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Example• Consider a notched tensile fatigue test on a titanium specimen.• The expected number of cycles to first acoustic emission(indicates crack initiation) is µ = 28, 000. The standarddeviation of the number of cycles is σ = 5, 000.• Let X1, . . . , X25be a random sample; each Xiis the numberof cycles on a different randomly selected specimen• Then, E(¯X) = µ = 28, 000 and the expected total number ofcycles for all 25 specimens is E T = nµ = 700, 000.• The standard deviations areσ¯X=σ√n=5, 000√25= 1000October, 2011Page 11Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011andσT=√nσ =√25(5, 000) = 25, 000October, 2011Page 12Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Normal Population Distribution Case• Let X1, . . . , Xnbe a random sample from a normal distributionwith mean value µ and standard deviation σ. Then for any n,¯Xis normally distributed with mean µ and standard deviationσ√n.• Note that this is true no matter what n is. It need not go toinfinity.October, 2011Page 13Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Linear combination of the random variables• Given a collection of n random variables X1, . . . , Xnandconstants a1, . . . , an, the RVY =nXi=1aiXiis called a linear combination of Xi’s•¯X is a special case with a1= . . . = an=1nwhile the total Tis another special case with a1= . . . = an= 1October, 2011Page 14Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Properties of linear combinations of random variables• Let X1, X2,. . .,Xnbe random variables with meansµ1, . . . , µnand variances σ21, . . . , σ2nrespectively.1. EPni=1aiXi=Pni=1aiµi2. If X1, . . . , Xnare independent,V (Pni=1aiXi) =Pni=1a2iσ2iOctober, 2011Page 15Statistics 511: Statistical MethodsDr. LevinePurdue UniversityFall 2011Example• A gas station sells regular, extra and super gasoline. The pricesare 3.00, 3.20 and 3.40 per gallon. Let X1, X2, X3be theamounts purchased on a particular day (in gallons).• Let X1, X2, X3be independent with µ1= 1000, µ2=