CHAPTER SIX some fundamental limit theorems Limit theorems characterize the mass behavior of experimental out- comes resulting from a large number of perforrna.nces of an experiment. These theorems provide the connection between probability theory and the measurement of the parameters of probabilistic phenomena in the real world. Early in this chapter, we discuss stochastic convergence, one important type of convergence for a sequence of random variables. This concept and an easily derived inequality allow us to establish one form of the law of large numbers. This law provides clarification of our earlier speculations (Sec. 3-6) regarding t,he relation, for large values of n, between the sum of n independent experimental ualues of random variable x and the quantity nE(x). We then discuss the Gaussian PDF. Subject to certain restric- tions, we learn that the Gaussian PDF is often an excellent approxima-tion bo the actual PPD for the sum of many random variables, regardless of the forms of the PDF7s1for the individual random variables included in the sum. This altogether remarkable result is known as the central limit - a theorem. A proof is presented for the case where the sum is composed ot independent identically distributed random variables. Finally, we investigate several practical approximation procedures based on limit t heorenzs. 6-1 The Chebyshev Inequality The Chebysher~ inequality shtes an upper bound on the probability that an experimental value of any random variable z will differ by at least any given positive quantity t from E(x). In particular, the inequality will provide an upper bound on the quantity in t,erl>ls of t and o,. As long as the value of the standard deviation rr, is known, other details of the PDF f,(xo) are not relevant. The derivation is simple. With t > 0, we have To obtain the above inequality, n7e note that the integrand in the left- most integration is always positive. BY removing an interval of length 2t from the range of that integral, n-e cannot increase the value of the integral. Inside the two integrals on the right-hand side of the above relat,ion, it is always true that 1x - lC(x)( 2 t. We now replace [x - E(z)lZ by t2, which can never increase the valuc of the right-hand side, resulting in E(x)-t gzZ 2 jz0 = - t2&(xo) ~XQ + /xy=e(z)+t t2b(xo) ~XO After we divide both sides by t2 and rec,ogllize the physical interpreta- t,ion of the remaining quantity on the right-hand side, we have - e = - - - - = Pmb[r - E(z)\ 2 t] 5 @y - - which is the Chebyshev inequality. It states, for instance, that the probability that an experinwntal value of any random variable x will be further than Kc, from E(x) is always less than or equal to 1/K2 Since it is a rather ~veali bound, the Chebyshev inequality finds most of its applications in general theoretical work. For a random variable described by any particular I'UI", better (though usually more complex) bounds may he est:iblishcd. We shall use the Chebyshev bound in Sec. 6-3 to invcstig~ite one form of the lsrv of lnrgc numbers. 6-2 Stochastic Convergence A dete~winistic sequence (1,,/ = .r,, .et, . . . is said to converge to t*he limit C if for every E > 0 wc can find a finite no such that Ix,~ - CI < E for all n > no If dctcrministic sequence (r,] does converge t,o thc limit C, 11-c write Only for pathological cases would we expect to be able to make equally strong nonprobnbilistic convergence statements for sequences , of random variables. Several different types of convergerm are defined for sequences of randon1 variables. In this section we introduce and discuss one such definition, namely, that of stochastic conuo-gence. We shall use this definition and the Chebyshev inequality to establish a form of the law of large numbers jin the following section. (We defer any discussion of other fornts of eorivergeuec for sequences of random variables unt,il Sec. G-9.) - - - - - iC-- - P A sequence of random variables, (g./ = gr, y2, y3, . . . , is said - = - to bc stochasticallg convergent (or to converge in probability) to C if, for ESZ = - every E > 0, the condition linl I'rob(/y, - C/ > e) = 0 is satisfied. n--+ rn - - When a sequence of rahdorn variables, (g,J, is linorvn to be stochastically convergent to C, we must be careful to conclude onlg that the probability of the event ly, - C/ > E vanishes as n m. We cannot conclude, for any value of n, that this event is impossible. We may use the definition of a limit to restate the definition of stochastic convergence. Sequence { y, 1 is stochastically convergent to C if, for any F > 0 and any 6 > 0, it is possible to state a finite value of no such that i'rob((y, - C( > E) < 6 for all n > no Further discussion will accompany an application of the concept- - - - - - - - for this equation there finally results m m-of stochastic convergence in the following section and a coniparison witshother forlxs of probabilistic convergence in Sec. 6-9. 6-3 The Weak Law of Large Numbers A sequence of random variables, (g, 1, with finite expected values, is said t,o obey a Inw oJ large numbers if, in some sense, the sequence defined by n is1 converges to its expected value. The type of convergence which applies determines whether the law is said to be weak or strong. Let 91,yz, . . . form a sequence of independent identically dis-tributed random variables with finite expected values E(y) and finite variances aV2. In this section me prove that the sequence is stochastically co~ivergentto its expected value, and therefore the sequence {g.] obeys a (weak) law of large numbers. For the c,onditions given above, random variable ill, is the average of n independent experimental values of random variable y. Quantity M, is known as the sample mean. From the definition of rtl, and the property of expectations of sums, we have and, because multiplying srandom variable y by cdefines a new random variable with a variance equal to c2oY2,we have To establish the weak law of large numbers for the case of interest, me simply apply the Chebyshev inequality to A1, to obtain and, substituting for M,, E(M,), and tor.., we find (See Prob. 6.02 for a practical application. of this relation.) Upon taking the limit as n --+ iirn Prob [I I; yi -B(9)/ 2 = 0 mn-+ co i=l m m mhich is known as the weak. law of large numbers (in this
View Full Document
Unlocking...