The Central Limit TheoremThe figure below shows the graphs of two random variables.• The fist random variable is the number of heads obtained after flipping a biased coin 30 times, wherethe chance of getting heads on a single flip is 3/4.• The second random variable is the sum of the numbers you get after rolling a (fair six-sided) die 10times.Notice how similar these plots look. Of course, these random variables have different expected values andstandard deviations, but their basic shape is the same. It’s something like this:This is rather astonishing. Just about the only thing these random variables have in common is that theycan be written as the sum of a bunch of independent random variables. What’s even more amazing is thatthis one property: “being the sum of a bunch of independent random variables” is enough to make any otherrandom variable look like this as well. This is essentially what the central limit theorem says.Before we continue, recall a couple rules about expected value and standard deviation. If a and b areconstants, and f is a random variable, thenE(af + b) = aE(f) + b and σ(af + b) = aσ(f )Now the expected value (which measures where the center of the distribution is) and standard deviation(which measures how spread out the distribution is) of our random variables can be anything. However,1we can remove these differences by considering the random variablef−E(f)σ(f)instead of f. This new randomvariable has expected value 0, and standard deviation 1: since E(f) and σ(f) are constants, by the two rulesabove,Efσ(f)−E(f)σ(f)=E(f)σ(f)−E(f)σ(f)= 0 and σf − E(f)σ(f)=σ (f − E(f ))σ(f)=σ(f)σ(f)= 1Removing different expected values and standard deviations this way turns out to be exactly the way whichmakes their distributions almost exactly the same. We’re ready to state the central limit theorem:The (Fuzzy) Central Limit Theorem:1If f is a random variable that’s the sum of lots of independentrandom variables: f = f1+ f2+ ···+ fn, then the probability thatf − E(f)σ(f)is between a and b (for a < b)is approximately the same for any such f, and is ≈Zbae−x2/2√2πdx. Note that the fidon’t all have to havethe same distribution, even though the examples we’ve given have this property.The functione−x2/2√2πis called a Gaussian, and this is the function that’s graphed on the previous page.Finding an antiderivative of this function is impossible (unless we use functions defined by integrals) whichis why we’ve leftRbae−x2/2√2πdx unsimplified. To do these integrals, you have to approximate them. You cando this on a calculator, in spreadsheets2, computer algebra systems3, from tables, at webpages (like the onein the class notes), and many other places.Let’s look at some basic properties. First, a “sanity check”: the probability thatf−E(f)σ( f )is between−∞ and ∞ should be 1, and indeedR∞−∞e−x2/2√2πdx = 1 (you can actually do this particular integral, butit involves some tricky vector calculus). Next, notice thate−x2/2√2πis even (it’s symmetric around 0). Thatmeans that the integralsRbae−x2/2√2πdx = 1 andR−a−be−x2/2√2πdx = 1 will be equal. So, for instance,P (f − E(f)σ(f)≥ r) ≈ P (f − E(f)σ(f)≤ −r) ≈Z∞re−x2/2√2πIn particularR∞0e−x2/2√2π=12, thus, the chance of f being greater than its expected value is approximately1/2. Also, by adding P (f−E(f)σ(f)≥ r) and P (f−E(f)σ(f)≤ −r) for r ≥ 0, we getP (|f − E(f)σ(f)| ≥ r) ≈ 2Z∞re−x2/2√2πIn the lecture notes, the central limit theorem was stated using this approximation.Now let’s do a few examples. As we’ll see, the central limit theorem has huge significance as a calculationaltool. It allows us to accurately estimate probabilities that would almost impossible to find otherwise; wedon’t have any other tools for estimating the probability that a random variable is between two numbersexcept for direct calculation.1Of course, what’s stated above isn’t actually a theorem. What the central limit theorem really says is that as we add moreand more independent random variables to f = f1+ f2+ · · · + fn, then the limit as n → ∞ of P“a ≤f −E(f )σ(f )≤ b”is exactlyRbae−x2/2√2πdx. What we’ve written above is a consequence of the central limit theorem that we’ll be using. Note also that thecentral limit theorem has some modest requirements on the fithat we haven’t stated (but which will be true in the problemswe do). For instance, they can’t conspire to make f attain one value with nonzero probability in the limit n → ∞.2For instance, in Excel, normdist(x,0,1,1) does the integral from −∞ to x3For instance, in Matlab, normcdf(x,0,1) does the integral from −∞ to x21. What’s the approximate probability of getting a sum between 25 and 40 when you roll 10 dice?Let’s call the random variable giving the sum of the dice f. The expected value of our random variableis E(f) = 35, and the standard deviation is σ(f) =q35012. Now25 ≤ f ≤ 40 ↔ −10 ≤ f − 35 ≤ 5 ↔−10q35012≤f − 35q35012≤5q35012So we need to estimate the probability that −1.852 ≤f−E(f)σ( f )≤ .926. By the central limit theoremthis is ≈R.926−1.852e−x2/2√2πdx ≈ .79. The actual probability is about .8182, so this is a fair estimate.2. On the last quiz, we considered flipping a fair coin labeled “-1” and “1”, 10,000 times. Let f bethe random variable giving the sum of all the numbers we get. Then E(f) = 0, and σ(f) = 100.Approximate the probability that |f | ≥ 1, 000, and compare it to the upper bound you found usingChebyshev’s theorem.By the values of E(f) and σ(f ) given above,|f| ≥ 1,000 ↔f − E(f)σ(f)≥ 10By the central limit theorem, P (f−E(f)σ(f)≥ 10) ≈ 2R∞10e−x2/2√2πdx ≈ 1.524 × 10−23. The probability isextremely small! Much much smaller than the upper bound of 0.01 we got using Chebyshev’s