MIT OpenCourseWare http://ocw.mit.edu 18.443 Statistics for Applications Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.� � 18.440 and 18.443 Gamma and beta probabilities This handout is based on section 1.5 of a book manuscript, Handbook and “Tables” of Classic Probabilities, by Robert J. Holt, R. M. Dudley, David Yang Gao, a nd Lewis Pakula. The numbering from that section is preserved, but some revisions have been made. 1.5 Gamma and beta functions and probabilities. The gamma function is defined for any a > 0 by ∞ (1.5.1) Γ(a) := x a−1 e −xdx. 0 The integral is finite if (and only if) a > 0, because �01 xa−1dx = 1/a < ∞, and xa−1 < ex/2 for x large enough. Integration by parts shows that Γ(a + 1) = aΓ(a) for any a > 0. We have Γ(1) = 1. It follows by induction that Γ(k + 1) = k! for any nonnegative integer k. For any a > 0 the function defined by (1.5.2) γa(x) := x a−1 e −x/Γ(a) for x > 0, and 0 for x ≤ 0, is a probability density. The corresponding distribution is called a gamma distribution with parameter a. If the random variable X has a gamma distributio n with parameter a then EX = a since EX = Γ(a + 1)/Γ(a). Likewise EX2 = Γ(a + 2)/Γ(a) = (a + 1)a so Var(X) = a and σX = a1/2 . Recall t hat for any random variable X with density f and any c > 0, cX has a density c−1f(x/c). Applying that to c = 1/λ for any λ > 0, if X has density γa then X/λ has the density γa,λ defined by γa,λ(x) = λa x x−1 e −λx/Γ(a) for 0 < x < +∞ and 0 otherwise. A random variable Y with this density evidently has EY = a/λ and Var(Y ) = a/λ2 . The Beta function is defined for any a > 0 a nd b > 0 by � 1 (1.5.3) B(a, b) := x a−1(1 − x)b−1dx. 0 Clearly, 0 < B(a, b) < ∞ for any a > 0 and b > 0. Letting y := 1 − x shows that B(b, a) ≡ B(a, b). Let βa,b(x) := xa−1(1 − x)b−1/B(a, b) for 0 < x < 1 and 0 for x ≤ 0 or x ≥ 1. Then βa,b is a probability density. The probability distribution with this density is called a beta distribution with parameters a , b. Its distribution function is then defined as x (1.5.4) Ix(a, b) := βa,b(t)dt, 0 ≤ x ≤ 1. 0 1� � � a The following fact relates gamma distributions wit h different parameters with each other and relates gamma and beta functions. 1.5.5 Theorem. For any a > 0 and b > 0, (a) B(a, b) ≡ B(b, a) ≡ Γ(a)Γ(b)/Γ(a + b). (b) If X and Y are independent random variables having gamma distributions with pa-rameters a and b respectively, then U := X + Y has a gamma distribution with parameter + b.Proof. First consider (b). U has a density u given by a convolution of those of X and Y , namely, for any x > 0, x u(x) = γa(x − y)γb(y)dy 0 x = (x − y)a−1 e −(x−y)y b−1 e −ydy/(Γ(a)Γ(b)) 0 x −x = e (x − y)a−1 y b−1dy/(Γ(a)Γ(b)). 0 The substitution y = tx, 0 ≤ t ≤ 1 gives −x = e x a+b−1B(b, a) /(Γ(a)Γ(b)). Since u must be a probability density, it must be the g amma density with parameter a +b, and the normalizing constants must agree, so both (a) and (b) follow. � Iterating Theorem 1.5.5, it follows that if Xi are independent identically distributed variables, each hav ing the standard exponential distribution with density e−x for x ≥ 0 and 0 for x < 0, so that the Xi have gamma distributions with parameter 1, then for each n = 1, 2, .. ., Sn = X1 + ... + Xn has a γn density. If each Xi has a γa,λ density then Sn has a γna,λ density. It is now easy to find the means and variances of beta distributions. If X has a beta distribution with parameters a, b, in other words has distribution function (1.5.4), then EX = B( a+1, b)/B(a, b). Similarly EX2 = B( a+2, b)/B(a, b) = a(a+1) /[(a+b)(a+b+1)]. Thus ab (1.5.6) EX = a/(a + b), Var(X) = . (a + b)2(a + b + 1) Note that 1 − X has a beta distribution with parameters b, a. Thus E(1 − X) = b/(a + b) which equals 1 − a/(a + b) as it should. Also, 1 − X has the same variance as X, and so the expression for Var(X) is preserved by interchanging a and b a s it should be. Let 0 < λ < ∞ and let Y be a Poisson random variable with parameter λ. Then some notations are k P (k, λ) = Pr(Y ≤ k) = e −λ �λj/j!, j=0 2�� ∞ −λ λj/j!.Q(k, λ) = Pr(Y ≥ k) = e There are identities relating the Poisson and gamma distributions: 1.5.7 Theorem. For any positive integer k, if X has a γk density, we have for any x ≥ 0, (1.5.8) Q(k, x) = P (X ≤ x) and (1.5.9) P (k − 1, x) = P (X > x). For 0 < λ < ∞, if Y has a γk,λ density and 0 < t < ∞, then (1.5.10) P (Y ≤ t) = Q(k, λt) �j k=(1.5.11) P (Y > t)P (k 1, λt)− . �and Proof. Equation (1.5.9) follows by differenti a ting with respect to x and noting that the derivative of P (k − 1, x) gives a (finite) telescoping sum. Equation follows by taking complements. Then letting Y = X/λ, Y has the given density, and (1.5.11) foll ows from ( 1.5.9), and (1.5.10) follows by taking complements or from (1.5.8). � A similar identity relates beta and binomial probabilities. Let 0 < p < 1, q = 1 − p, let X be a binomia l (n, p) random variable and k j=0 B(k, n, p) = Pr(X ≤ k) = b(j, n, p), E(k, n, p) = Pr(X ≥ k) = n j=k b(j, n, p). 1.5.12 Theorem. If 0 < p < 1, and 0 ≤ k ≤ n are integers, then E(k, n, p) = Ip(k, n − k + 1), if k ≥ 1; B(k, n, p) = I1−p(n − k, k + 1), if k < n. Proof. T he first equality again follows from differenti a ting a finite sum with respect to p which gi ves a telescoping sum. The second then foll ows from B(k, n, p) ≡ E(n−k , n, 1−p). 3� � � � � � � Proof. We have Quotients of independent variables with the densities just gi ven have distributions that can be expressed in terms of beta probabilities: 1.5.13 Theorem. Let X and Y be independent gamma variables