Statistics 550 Notes 10 Reading: Section 2.1.Take-home midterm: I will e-mail it to you by the morning of Saturday, October 14th. It will be due Monday, October 23rd by 5 p.m.I. Method of MomentsSuppose 1, ,nX XKiid from ( | ), }p x �Qq q{ where qis d-dimensional. Let 1( ), , ( )dm mKq qdenote the first d-moments of the population we are sampling from (assuming that they exist), ( ) ( ), 1jjE X j dm = � �qqDefine the jth sample moment ˆjmby11ˆ, 1 j dnjj iiXnm== � ��.The function 1 1ˆ ˆ( , ) ( ( ), , ( ))d dm m m m= - -Ky q q qXis an estimating equation for which 0 0 1 1ˆ ˆ( ( , ) ( ( ), , ( ))d dV E E Em m m m= = - - =Kθ X0 0 0q q q,q ) y q q q 0For many models, 0 0( 0 for all V � �θ ,q) q q so thatˆ( , ) 0=y qX is a valid estimating equation. 1Suppose 1( ( ), , ( ))dm m� Kq q qis a 1-1 continuous function from d�to d�. Then the estimating equation estimate of qbased on ( , )y qXis the ˆqthat solvesˆ( , ) =y qX 0, i.e.,ˆˆ( ) 0, 1, , .j jj dm m- = = KqExample 4: 1, ,nX XKiid Gamma( , )p l. 1( | , )( )p p xx ef x pplll- -=G for 0x >. (see Section B.2.2 of Bickel and Doksum).The first two moments of the gamma distribution are 2, ,2( 1)( ) , ( )p pp p pE X E Xl ll l+= =. (see Excercise B.2.3, page 526).The method of moments estimator solves212ˆ0ˆˆ ˆ( 1)0ˆniipXXp pnll=- =+- =�which yields 221ˆniiXXXnl==-� and 221ˆˆniiXp X XXXnl== =-�.2Example: The gamma model is frequently used for describing precipitation levels. In a study of the natural variability of rainfall, the rainfall of summer storms was measured by a network of rain gauges in southern Illinois for the years 1960-1964. 227 measurements were taken.For these data, 21.224, 0.184niiXXn== =�, so that the method of moments estimates are 322212221.224ˆ1.674.184 .2240.224ˆ0.224 0.3750.184 0.224niiniiXXXnXp XXXnl=== = =--= = =--��The following plot shows the Gamma (.375, 1.674p l= =)density plotted on the histogram. In order to make the visual comparison easy, the density was normalized to havea total area equal to the total area under the histogram, which is the number of observation times the bin width of the histogram, or 227*.2=45.4.4Qualitatively, the fit of the gamma model to the data looks reasonable; we will examine methods for assessing the goodness of fit of a model to data in Chapter 4.Large sample motivation for method of moments:A reasonable requirement for a point estimator is that it should converge to the true parameter value as we collect more and more information. Suppose 1, ,nX XKiid.A point estimator h(X1,...,Xn) of a parameter ( )q q is consistent if h(X1,...,Xn)( )Pq� qas n � �for all �Qq. 5Definition of convergence in probability (A.14.1, page 466). h(X1,...,Xn)( )Pq� q means that for all 0e >, 1lim [| ( ,..., ) ( ) | ] 0nnP h X X q e��- � =q. Under certain regularity conditions, the method of momentsestimator is consistent. We give a proof for a special case Let 1( ) ( ( ), , ( ))dg m m= Kq q q. By the assumptions in formulating the method of moments, gis a 1-1 continuous function from d�to d�. The method of moments estimator solves1ˆˆ ˆ( ) ( , , )dg m m- =Kq 0.When the g’s range is d�, then11ˆˆ ˆ( , , )dg m m-Kq =. We prove the method of moments estimator is consistent when 11ˆˆ ˆ( , , )dg m m-Kq = and 1g-is continuous. Sketch of Proof: The method of moments estimator solvesˆˆ( ) 0, 1, , .j jj dm m- = = KqBy the law of large numbers, 1 1ˆ ˆ( , , ) ( ( ), , ( ))Pd dm m m m�K Kq q.By the open mapping theorem (A.14.8, page 467), since 1g-is assumed to be continuous, 1 11 1ˆˆ ˆ( , , ) ( ( ), , ( ))Pd dg gm m m m- -= � =K Kq q q q6Comments on method of moments:(1) Instead of using the first d moments, we could use higher order moments (or other functions of the data – see Problem 2.1.13) instead, leading to different estimating equations. But the method of moments estimator may be altered by which moments we choose.Example: 1, ,nX XKiid Poisson(l). The first moment is 1( ) ( )E Xlm l l= =. Thus, the method of moments estimator based on the first moment is ˆXl =. We could also consider using the second moment to form a method of moments estimator. 2 22( ) ( )E Xlm l l l= = +.The method of moments estimator based on the second moment solves2 211ˆ ˆniiXnl l== +�Solving this equation (by taking the positive root), we find that 1/ 2211 1 1ˆ2 4niiXnl=� �=- + +� �� ��.The two method of moments estimators are different.For example, for the data > rpois(10,1) [1] 2 3 0 1 2 1 3 1 2 1,7the method of moments estimator based on the first moment is 1.1 and the method of moments estimator based on the second moment is 1.096872.(2) The method of moments does not use all the information that is available. 1, ,nX XKiid Uniform(0, )q. The method of moments estimator based on the first moment is ˆ2Xq =. If 2 maxiX X<, we know thatˆmaxiXq q� >II. Minimum Contrast heuristicMinimum contrast heuristic: Choose a contrast function( , )r X qthat measures the “discrepancy” between the data X and the parameter vector q. The range of the contrast function is typically taken to be the real numbers greater than or equal to zero and the smaller the value of the contrast function, the more “plausible” qis based on the data X. Let 0qdenote the true parameter. Define the population discrepancy 0( , )D q qas the expected value of the discrepancy ( , )r X q:00( , ) ( , )D E r�qq q qX8In order for ( , )r X qto be a valid contrast function, we require that 0( , )D q qis uniquely minimized for 0q =q, i.e.,0 0 0 0( , ) ( , ) if D D> �q q q q q q.0q =qis the minimizer of 0( , )D q q. Although we don’t know 0( , )D q q, the contrast function ( , )r X qis an unbiased estimate of 0( , )D q q (see ). The minimum contrast heuristic is to estimate qby minimizing ( , )r X q, i.e., ˆmin ( , )r�qq = qXQ.Example 1: Suppose 1, ,nX XKiid Bernoulli (p), 0 1p< <. The following is an example of a contrast functions and an associated estimate:“Least Squares”: 21( , ) ( )niip X pr== -�X.020120 0( , ) [ ( ) ] 2np iiD p p E X pnp npp np== - == - +�We have00( , )2 2D p pnp npp�=- +�and it …