Week 3Spring 2009Lecture 5. Bayes estimation, minimaxity and Admissibility (cont.).AdmissibilityConditions on priors and admissibility: conditions on the prior measurewhich guarantees that the corresponding generalized Bayes procedure is admis-sible.De…neJx(h) =Zh () ' (x ) d.Let S1= fx 2 Rp: kxk 1g.AssumptionGrowth Condition:ZSc1g ()kk2log2(kk)< 1Asymptotic ‡atness condition:ZSc1Jx(rggJx(rg)Jx(g)2g)dx < 1It can be shown thatRSc1krgk2=gdx < 1 implies the ‡atness condition.Theorem. Let G be a prior satisfying two conditions above. Then Gisadmissible.For the normal mean estimation problem with squared error loss,Blyth’s Method. Let be an estimator. Let fGjg be a sequence of …-nite prior measures such that:(i) r(Gj; ) r(Gj; Gj) ! 0 as j ! 1; (ii)infjfGj(S1)g > 0. Then is an admissible estimator.hint : Let 00=0+ =2. If R; 0 R (; ) for all and with strictinequality for some , then R; 00< R (; ) for all . For all j;r(Gj; Gj) r(Gj; 00) ZS1R; 00Gj(d) +ZSc1R (; ) Gj(d)= r(Gj; ) +ZS1R; 00 R (; ) Gj(d) r(Gj; ) "where " =RSR; 00 R (; ) Gj(d). Contradiction!Example. Let X N (; 1). Let gj(x) =1p2expx22j, and gj=dGj=d where is the Lebe sgue measu re. It is easy to showr(Gj; X) =pj, r(Gj; Gj) =pjjj + 11thenr(Gj; X) =pjj + 1! 0.Proposition.r(G; ) r(G; G) =ZkG k2g (x) dxProof :r(G; ) r(G; G) = EXEk G+ G k2 k Gk2jX= EXEkG k2jX=ZkG k2g (x) dxProof of the theorem: Please read page 374 of Stein (1961).De…ne gj= h2jg wherehj=8<:1 kk 11 log(kk)log j1 kk j0 kk > j, j = 2; 3; : : : .It is easy to seeGj=Rh2jg () ' (x ) dRh2jg () ' (x ) d! Ga.s.andg j(x) =Zh2jg () ' (x ) d g (x) .WriteG(x) = x +rg g = x +Jx(rg)Jx(g);Gj(x) = x +rg g = x +Jxh2jrg + grh2jJxh2jgwhere the second equality for each equation follows from integration by parts.Hencer(Gj; G) r(Gj; Gj)=ZSc1G Gj2g j(x) dx +ZSc1G Gj2g j(x) dx ( apply DCT, since g j(x) g (x) …nite.) 2ZSc1Jxgrh2jJx(gj)2g j(x) dx + 2ZSc1Jx(rg)Jx(g)Jxh2jrgJx(gj)2g j(x) dx + o (1)= 2Aj+ 2Bj+ o (1)2Show Aj! 0 by DCT:Aj=ZSc1Jxgrh2jJx(gj)2g j(x) dx= 4ZSc1Jx(ghjrhj)Jx(gj)2g j(x) dx = 4ZSc1Jxg1=2j g1=2jrhjJx(gj)2g j(x) dx 4ZSc1Jxg krhjk2dx (Cauchy-Schwartz inequality and gj g) 4ZSc1krhj()k2g () dandkrhj()k2=1kk2log2(j)I[1;j](kk) 1kk2log2(kk_ 2)I[1;j](kk) .Show Bj! 0 by DCT again:Jx(rg)Jx(g)Jxh2jrgJx(gj)2g j(x) (Note that h2jrg is gjrgg)=JxgjJx(rg)Jx(g ) h2jrg2Jx(gj)=JxhgjJx(rg)Jx(g )rggi2Jx(gj) Jx gjJx(rg)Jx(g)rgg2!(Cauchy-Schwartz inequality) Jx gJx(rg)Jx(g)rgg2!.Admissibility of 0= X for p= 1; 2Let g () = 1, thenZSc1g ()kk2log2(kk)= 2Z121r log2rdr < 1.Homework problem (you pick one part to work on). Let Xi Poisson (i)be independent, i = 1; 2; : : : ; p. Denote X = (X1; : : : ; Xp) and = (1; : : : ; p).Under the loss L (; ) =Ppi=1(i i)2=i, show that (i) for p = 1, X is anadmissible estimator of using Blyth’s method; (2) for p 2, X is not anadmissible estimator of .3Reference: (i) Clevenson and Zidek (1975), Simultaneous Estimation of theMeans of Independent Poisson Laws, JASA.(ii) Brown and Hwang (1982), A uni…ed admissibility proof, Statistical De-cision Theory and Related Topics, III. S. S. Gupta and J. O. Berger (eds.)4Lecture 6. Bayes estimation, minimaxity and Admissibility (cont.).Superharmonic priors and minimaxity.De…nition. Let h : Rp! R be twice di¤erentiable. We call h super-harmonic ifr2h (x) 0 for all x 2 Rp, where r2h (x) =pXj=1@2@x2ih (x)and call h harmonic if r2h (x) = 0 for all x 2 Rp. The operator r2is calledLaplace operator (other notations include and r r).Let X N (; I). Let G be any prior such that g (x) < 1 for all x. Thenthe generalized Bayes estimator GisE (jX = x) = x + r(log (g (x))) .By Ste in’s unbiased estimate of the risk, we haveR (; G) p = E(2r rg g +rg g 2)= E(2r2g g rg g 2).Let h =pg . We seeR (; G) p = 4Er2h h (or 4Er2pg pg ).sinceE(2r rg g +rg g 2)= 4E(r rh h (x)+rh h 2)= 4E(r2h h krh k2(h )2+rh h 2)Theorem. If r2pg 0 (or r2g ), then Gis minimax.SinceR (; G) p = E(2r2g g rg g 2).andr2g (x) =Zr2g () ' (x ) d,then Gis min imax if g is super-harmonic.Calculation of the Harmonic Bayes procedure.5Letg () = 1= kkp2This prior density is called the “harmonic prior”because it is harmonic at everypoint except = 0. Note that it is a a valid generalized prior density whenp 3 in the sense that g (x) < 1 for all x . And it is true that g (x) issuperharmonic.Writeg () = 1= kkp2_Z10!(p4)=2e!kk2=2d!=Z101v2v1 vp=2evkk22(1v)dvtheng (x) _Z10v(p4)=2exp kxk2v2!dv,and thusrg g =R10vp=21expkxk2v2dvR10vp=22expkxk2v2dvx =1kxk2Rkxk20yp=21exp (y=2) dyRkxk20yp=22exp (y=2) dyxWe haveG(x) = x + r(log (g (x))) = 1 p 2kxk21 ekxk2=2Tp+2(kxk2)1 ekxk2=2Tp(kxk2)!xwhereTpkxk2=(p=22k=0(s=2)k=k! p 4 even(p1)=21k=02kk!s2k+1= (2k + 1)! p 3
View Full Document