DOC PREVIEW
UCLA STAT 100A - MGF Derivations Discrete

This preview shows page 1-2 out of 6 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

4 Moment generating functionsMoment generating functions (mgf) are a very powerful computational tool.They make certain computations much shorter. However, they are only acomputational tool. The mgf has no intrinsic meaning.4.1 Definition and momentsDefinition 1. Let X be a random variable. Its moment generating functionisMX(t) = E[etX]At this point in the course we have only considered discrete RV’s. Wehave not yet defined continuous RV’s o r their expectation, but when we dothe definition of the mgf for a continuous RV will be exactly the same.Example: Let X be geometric with parameter p. Find its mgf.Recall that fX(k) = p(1 − p)k−1. ThenM(t) =∞Xk=1etkp(1 − p)k−1= pet∞Xk=1et(k−1)(1 − p)k−1= pet11 − et(1 − p)Note that the geometric series that we j ust summed only converges ifet(1 − p) < 1. So the mgf is not defined for all t.What is the point? Our first application is show that you can get themoments of X from its mgf (hence the name).Proposition 1. Let X be a RV with mgf MX(t). ThenE[Xn] = M(n)X(0)where M(n)X(t) is the nth derivative of MX(t).Proof.dndtnE[etX] =dndtnXketkfX(k) =XkknetkfX(k)At t = 0 this becomesXkknfX(k) = E[Xn]1There was a cheat in the proof. We interchanged derivatives and aninfinite sum. You can’t always do this and to justify doing it in the abovecomputation we need some assumptions on fX(k). We will not worry aboutthis issue.Example: Let X be binomial RV with n trials and probability p of success.The mgf isE[etX] =nXk=0etknkpk(1 − p)n−k=nXk=0nk(pet)k(1 − p)n−k= [pet+ (1 − p)]nNow we use it to compute the first two moments.M′(t) = n[pet+ (1 − p)]n−1pet,M′′(t) = n(n − 1)[pet+ (1 − p)]n−2p2e2t+ n[pet+ (1 − p)]n−1petSetting t = 0 we haveE[X] = M′(0) = np, E[X2] = M′′(0) = n(n − 1)p2+ npSo the var ia nce isvar(X) = E[X2] − E[X]2= n(n − 1)p2+ np − n2p2= np − np2= np(1 − p)4.2 Sums of independent random variablesSuppose X and Y are independent random varia bles, and we define a newrandom variable by Z = X + Y . Then the pmf of Z is given byfZ(z) =Xx,y:x+y=zfX(x)fY(y)The sum is over all points (x, y) subject to the constraint that they lie onthe line x + y = z. This is equivalent to summing over all x and settingy = z − x. Or we can sum over all y and set x = z − y. SofZ(z) =XxfX(x)fY(z − x), fZ(z) =XyfX(z − y)fY(y)2Note that this formula look like a discrete convolution. One can use thisformula to compute the pmf of a sum of independent RV’s. But computingthe mgf is much easier.Proposition 2. Let X and Y be independent random variables. Let Z =X + Y . Then the mgf of Z is given byMZ(t) = MX(t)MY(t)If X1, X2, · · · , Xnare independent and identically distributed, thenMX1+X2+···+Xn(t) = [M(t)]nwhere M(t) = MXj(t) is the common mgf of the Xj’s.Proof.E[etZ] = E[et(X+Y )] = E[etXetY] = E[etX] E[etY] = MX(t)MY(t)The proof for n RV’s is the same.Computing the mgf does not give you the pmf of Z. But if you get a mgfthat is already in your catalog, then it effectively does. We will illustratethis idea in some examples.Example: We use the proposition to give a much shorter computation ofthe mgf of the binomial. If X is binomial with n tr ia ls and probability p ofsuccess, then we can write it as a sum of the outcome of each trial:X =nXj=1Xjwhere Xjis 1 if the jth trial is a success and 0 if it is a failure. The Xjareindependent and identically distributed. So the mgf of X is that of Xjraisedto the n.MXj(t) = E [etXj] = pet+ 1 − pSoMX(t) =pet+ 1 − p n3which is of course the same result we obtained before.Example: Now suppose X and Y are indep endent, both are binomial withthe same probability of success, p. X has n tr ia ls and Y has m trials. Weargued before that Z = X + Y should be binomial with n + m trials. Nowwe can see this from the mgf. The mgf of Z isMZ(t) = MX(t)MY(t) =pet+ 1 − p npet+ 1 − p m=pet+ 1 − p n+mwhich is indeed the mgf of a binomial with n + m trials.Example: Look at the negative binomial distribution. It has two parametersp and n and the pmf isfX(k) =k − 1n − 1pn(1 − p)k−n, k ≥ nSoMX(t) =∞Xk=netkk − 1n − 1pn(1 − p)k−n=∞Xk=netk(k − 1)!(n − 1)!(k − n)!pn(1 − p)k−nLet j = k − n in the sum to get∞Xj=0et(n+j)(n + j − 1)!(n − 1)!j!pn(1 − p)j=etnpn(n − 1)!∞Xj=0(n + j − 1)!j!etj(1 − p)j=etnpn(n − 1)!∞Xj=0dn−1dxn−1xn+j−1|x=et(1−p)=etnpn(n − 1)!dn−1dxn−1∞Xj=0xn+j−1|x=et(1−p)The natural thing to do next is factor out an xn−1from the series to turnit into a geometric series. We do something different that will save some4computation later. Note that the n − 1th derivative will kill any term xkwith k < n − 1. So we can replace∞Xj=0xn+j−1by∞Xj=0xjin the a bove. So we haveetnpn(n − 1)!dn−1dxn−1∞Xj=0xj|x=et(1−p)=etnpn(n − 1)!dn−1dxn−111 − x|x=et(1−p)=etnpn(n − 1)!(n − 1)!11 − x|x=et(1−p)=etp1 − et(1 − p)nThis is of the form something to the n. The something is just the mgf ofthe geometric distribution with para meter p. So the sum of n independentgeometric random variables with the same p gives the negative binomial withparameters p and n.4.3 Other generating functionsThe book uses the “probability generating function” for r andom variablestaking values in 0, 1, 2, · · · (or a subset thereof). It is defined byGX(s) =∞Xk=0fX(k)skNote that this is just E[sX], and this is our mgf E[etX] with t = ln(s).Anything you can do with the probability generating function you can dowith the mgf, a nd we will not use the probability generating function.The mgf need not be defined for all t. We saw an example of this withthe geometric distribution where it was defined only if et(1 − p) < 1, i.e,t < − ln(1 − p). In fact, it need not be defined f or any t other than 0. Asan example of this consider the RV X that takes on all integer values andP (X = k) = c(1 + k2)−1. The constant c is given by1c=∞Xk=−∞11 + k25We leave it to the reader to show that∞Xk=−∞etk11 + k2= ∞for all nonzero t.Another moment generating function that is used is E[eitX]. A probabilistcalls this the charateristic function of X. An analyst might call it the fouriertransform of the distribution of X. It has the advantage that for real t it isalways defined.End of September 28


View Full Document

UCLA STAT 100A - MGF Derivations Discrete

Download MGF Derivations Discrete
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view MGF Derivations Discrete and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view MGF Derivations Discrete 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?