762notes.pdfCHAPTER 11 ST 762, M. DAVIDIAN11 The role of estimating weights in GLS – second ord er theory11.1 IntroductionThe “folklore” theorem for GLS discussed in Chapter 9 is used routinely as the basis for assessinguncertainty of GLS estimation (e.g. standard errors, confidence intervals, etc) for estimation of β (withtrue value β0) in the modelE(Yj|xj) = f(xj, β), var(Yj|xj) = σ2g2(β, θ, xj). (11.1)Recall that the main implications are as follows.• If the variance function g is correctly specified, how one estimates β and θ appearing in the“weights” does not affect the large-sample properties ofˆβGLSin the sense thatn1/2(ˆβGLS− β0)L−→ N (0, σ20ΣW LS), (11.2)where ΣW LSis the same as the m atrix that would arise if the weights were in fact known.• Regardless of the number of iterations of the GLS algorithm, C, (11.2) holds. Thus, the theoremoffers n o insight into how one should select C in pr actice; according to the theorem, the choice ofC does not matter.Recall also that it has been observed that using (11.2) as the basis for deriving estimated standarderrors may result in unreliable inferences. It has been noted that the standard errors obtained this waytend to understate the variability associated withˆβGLS, particularly for smaller n.The bottom line is that the folklore th eorem, despite its widespread use and “folklore” status, may notalways be that useful a result in practice.• The folklore theorem, and in fact all the large-sample distributional results we have discussed sofar, are first order results. That is, usual asymptotic normality results represent only a certainlevel of approximation with respect to n. In terms of the covariance matrix of n1/2(ˆβGLS− β0),the result tells us only thatvar{n1/2(ˆβGLS− β0)} = nvar{(ˆβGLS− β0)} ≈ σ20ΣW LS, (11.3)where ΣW LSis the (well behaved) limit of a quantity in the form of an average and where thismoment is unders tood as in all our arguments to be conditional on the xj.PAGE 272CHAPTER 11 ST 762, M. DAVIDIANWritten another way, (11.3) only tells us thatvar(ˆβGLS− β0) ≈ n−1σ20ΣW LS. (11.4)• For the model in (11.1) and GLS estimation, the level of first order approximation in (11.3) isnot sufficiently refined so that the effect of estimation of β and θ in the weights and number ofiterations C shows up.• Moreover, because of this, the approximation (11.4) may not yield a reliable representation of thetrue uncertainty when n is not too large.RESULT: A more refined approximation is needed. Loosely speaking, what is needed is somethingalong the lines ofvar{n1/2(ˆβGLS− β0)} ≈ σ20ΣW LS+ “other stuff”, (11.5)where, intuitively, “other stuff” must depend on n in such a way that it is “small” when n is “large,” infact “smaller” than the leading term σ20ΣW LS, so that the leading term dominates. This must be true;otherwise, “other stuff” would have shown up in the first order results. However, if n is n ot too large,“other stuff” might be nontrivial.In particular, (11.5) may also be written asvar(ˆβGLS− β0) ≈ n−1σ20ΣW LS+ n−1“other stuff”, (11.6)so it must be that the s econd term involving “other stuff” in (11.6) is o(n−1). Hopefully, “other stuff”is in part determined by the effects of estimation of β and θ in the weights and the choice of C, effectsthat d o not show up otherwise, so understanding its form may yield insight into the consequences ofestimating weights and h ow to choose C in practice. Presu mably, if we could obtain the form of “otherstuff,” we could use it to calculate more reliable estimated s tandard errors for practical use.SECOND O RD ER RESULTS: What we will find is th at “other stuff” for our problem turns out to beO(n−1) so that, in (11.6) we havevar(ˆβGLS− β0) ≈ n−1σ20ΣW LS+ n−2“stuff”,where we will determine the form of “stuff.” Such a repr esentation called a second order result; here,we have an approximation to the covariance matrix ofˆβGLSthat involves not only the leading termthat is O(n−1) but one th at is O(n−2) as well (the “second order” term).Generally, arguments to establish such second ord er results are very tedious. Consequently, in Sec-tion 11.2, we will pursu e such an argument in a simple, special case of (11.1).PAGE 273CHAPTER 11 ST 762, M. DAVIDIANEven in this situation, th e calculations are rather involved. In Section 11.3, we will simply state theresults of a more general argument given by Carroll, Wu, and Ruppert (1988).Throughout, we will assume that the variance function g in (11.1) is not misspecified, as our focus ison u nderstanding the performance of the first order results and how it might be improved upon whenthe model is correct.It turns out that such second order results, although providing some theoretical ins ight, do not translateinto improvements that may be used in p ractice, as the necessary calculations are much too difficult tobe implemented easily. In Section 11.4, we will consider use of the bootstrap for our model (11.1) as analternative way of effecting the same sort of impr ovement “automatically” under certain circumstances.11.2 Covariance matrix ofˆβGLSwhen g does not depend on βThe argument in this section is b ased on that of Rothenberg (1984). As usual, let β0, θ0, and σ0denotethe true values of these parameters, and suppress conditioning on the xjfor simplicity. The argumentuses the following restrictive assumptions, which we adopt for th is section only.• g does not depend on β (so only depends on an unknown variance parameter to be estimated).• The variance parameter θ is a scalar.• E(Yj|xj) = xTjβ; that is, the m ean model is linear in β, so that the fu ll mean-variance model isE(Yj|xj) = xTjβ, var(Yj|xj) = σ2g2(θ, xj) = σ2w−1j(θ). (11.7)• The conditional distribution of Yjgiven xjis normal, so thatǫj=Yj− xTjβ0σ0g(θ0, xj)= wj(θ0)(Yj− xTjβ0)/σ0∼ N (0, 1).The assumptions of linearity and normality actually may be relaxed, b ut simplify the argument sub-stantially; in particular, these assumptions allowed Rothenberg (1984) to make clever use of sufficiency,as we will demonstrate momentarily.Letˆθ be any estimator for θ such that(a) the distribution ofˆθ does not depend on β(b)ˆθ is an even function of ǫ1, . . . , ǫn(see below).PAGE 274CHAPTER 11 ST 762, M. DAVIDIANThese assumptions are actually quite reasonable: for