MIT OpenCourseWare http://ocw.mit.edu 18.443 Statistics for Applications Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.18.443 A LIKELIHOOD RATIO TEST FOR NESTED COMPOSITE HYPOTHESES: WILKS’S THEOREM Suppose we have a family of probabil ity density or mass functions f (x, θ) > 0 depend-ing o n a d-dimensional parameter θ. It will be assumed that there are partial derivatives through second order, ∂ log f(x, θ)/∂θi and ∂2 log f(x, θ)/∂θi∂θj for i, j = 1, ..., d, such that t he d × d matrix of second partials (“Hessian”) is a non-singular matrix for almost all x. This will assure that the parameter vector is truly d-dimensional. For example, we could have the family of all normal distributions N(µ, σ2), with d = 2. Let H1 be the set of possible parameters θ. In the normal case, µ can be any real number, and σ or σ2 any number > 0. Let H0 be an m-dimensional subset of H1 for some m < d. It wil l be assumed that H0 is “smoot h” in the sense that at any point of H0, we can select m of the parameters, say for example θ1, . .., θm, for which the other d − m parameters are twice differentiable functions of θ1, . .., θm. For example, H0 could be the intersection with H1 of an m-dimensional flat hyperplane. Assume that observations X1, . . . , Xn are i.i.d. with density f(x, θ) for some θ ∈ Θ. We want to test the hypothesis t hat θ ∈ H0. Let L(θ) = Πnj=1f( Xj , θ) be the likelihood function. Let M Ld be the maximum of the likelihood for θ in H1, in other words MLd = L(θˆd) where θˆd is the MLE of θ in H1, provided it exists. Let M Lm be, likewise, the maximum of the likelihood for θ i n H0. Then M Lm ≤ MLd because H0 ⊂ H1. Let Λ be the likelihood ratio, Λ = MLm/MLd, so that 0 < Λ ≤ 1. S. S. Wilks in 1938 proposed the following test: let W = −2 lo g Λ, so that 0 ≤ W < ∞. Wilks found that if the hypothesis H0 is true, then the distribution of W converges as n → ∞ to a χ2 distribution with d − m degrees of freedom, not depending on the true θ = θ0 ∈ H0. Thus, H0 would be rejected if W is too large in terms of the tabulated χd2 −m distribution. The l ikelihood ratio test of a multinomial hypothesis for k categories (Rice, Section 9.6) is a special case of Wilks’s test with d = k − 1. The fact that Wilks’s sta tistic has the given asymptotic distribution if H0 is true is called Wilks’s theorem. It holds under some hypotheses, not all of which have been stated, but which are given in references as indicated in the Notes. NOTES Wilks first published his theorem in a paper, Wilks ( 1938), then gave an exposition of it in his book, Wilks (1962, §13.8). Chernoff (1954) gave another proof. Van der Vaart (1998, Chapter 16) gives a more recent exposition. The Notes by van der Vaart (1998 , p. 240) suggest that Wilks’s ori ginal proof was not rigorous. The proof in the 1962 book seems rather long. A proo f is given in Dudley (2003), Section 3.9. REFERENCES Chernoff, Herman (1954). On the distribution of the likelihood ra tio statistic. Ann. Math. Statist. 25, 573-578. 1Dudley, R. M. (200 3). Mathematical Statistics, 18.466 lecture notes, Spring 2003. On MIT OCW (OpenCourseWare) website, 2004. van der Vaart , A. W. (1998). Asymptotic Statistics. Cambridge University Press. Wilks, S. S. (1938). The large-sample distribution of the likelihood ratio for testing com-posite hypotheses. Ann. Math. Statist. 9, 60-62. Wilks, S. S. (1962). Mathematical Statistics. Wiley, New York; 2d printing, corrected, 1963.