14.385Nonlinear EconometricsLecture 3.Theory : Consistency Continued.Asymptotic Distribution of Extremum Estima-tors1Cite as: Victor Chernozhukov, course materials for 14.385 Nonlinear Econometric Analysis, Fall 2007. MIT OpenCourseWare(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].1. MLE and M-Estimator ConsistencyHere we apply extremum consistency theoremto establish consistency of MLE and other M-estimators.Theorem 1 (MLE Consistency): If ziis i.i.d.with pdf f(z|θ) and (i) (Identification) f(zi|θ) =f(zi|θ0) with positive pro b ab i l i ty, for all θ = θ0;(ii) (Compactness.) Θ is co mpac t; (iii) (Con-tinuity) f(z|θ) is continuous at all θ with proba-bility one; (iv) (Dominance) E[supθ∈Θ|ln f(z|θ)pˆ|] <∞; then θ → θ0.In MLE we have θˆ∈ arg infˆθ∈ΘQ(θ), whereQˆ(θ) = −En[ln f(zi, θ)].By the LLN, the limit of the MLE objectivefunction will be Q(θ) = −E[ln f(z|θ)]. The fol-lowing result shows that the iden tific atio n con-dition in condition (i) is sufficient for E[ln f(z|θ)]2Cite as: Victor Chernozhukov, course materials for 14.385 Nonlinear Econometric Analysis, Fall 2007. MIT OpenCourseWare(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].66to be uniquely maximized at the true p arame-ter.Lemma 1 ( Information inequality): IfZ|ln f(z|θ)|f(z|θ0)dz < ∞for each θ ∈ Θ then if f(z|θ) = f(z|θ0) wehaveE ln[f(z|θ)] < E ln[f(z|θ0)]Proof: By the strict version of Jensen’s in-equality and concavity of ln(v),Zln[f(z|θ)/f(z|θ0)]f(z|θ0)dz (1)< ln{Z[f(z|θ)/f(z|θ0)]f(z|θ0)dz} (2)= lnZf(z|θ)dz = 0 . (3)It is interesting to note that identification ofθ0, in the sense that changing the parameter6Cite as: Victor Chernozhukov, course materials for 14.385 Nonlinear Econometric Analysis, Fall 2007. MIT OpenCourseWare(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].changes the density, is sufficient condition forQ(θ) = E[ln f(z|θ)] to have a unique maximumat θ0. For other extremum problems, it is oftenharder to give such simple suffici e n t con d i tio nfor identifiability.Proof of MLE consistency: I t suffices tocheck conditions of Extremum Consistency The-orem. For identification condition (i) let Q(θ) =E[ln f(z|θ)]. By the information inequality, forθ = θ0,Q(θ) − Q(θ0) = E[ln{f(z|θ)/f(z|θ0)}] < 0,giving identification. Compactness condition(ii) is assumed. Continuity condition (iii) anduniform convergenc e c o n d i tio n ( i v ) h o l d by theULLN of L2 applied to Qˆ(θ) = En[ln f(zi|θ)].Q.E.D.Binary Choice Continued: The probit (con-ditional) likeli h ood for a single observation z =Cite as: Victor Chernozhukov, course materials for 14.385 Nonlinear Econometric Analysis, Fall 2007. MIT OpenCourseWare(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].6(y, x) where y ∈ {0, 1} isf(z|yθ) = Φ(x0 1θ) [1 − Φ(x0θ)]−y,where Φ(v) is the standar d normal CDF.Theore m on Consistency of probit: If E[xx0]is finite and non sin g u l ar then the pro b i t MLEpθˆsatisfies θˆ→ θ0.Proof: For the standard normal pdf φ(v), it is wellknown that ∂ ln Φ(v)/∂v = φ(v)/Φ(v) is decreasing, sothat ln Φ(v) is concave. Also, ln[1 − Φ(v)] = ln Φ( v)is concave. Then, since a concave function of a−lin-ear function is concave, ln f(z θ) = y ln Φ(x0β) + (1y) ln[Φ(−x0θ)] is concave. Ther|efore, to show consis-−tency, it suffices to verify the conditions (i) - (iii) ofTheorem on Consistency of Argmin Estimators withConvexity.i) By nonsingularity of E[xx0], for θ = θ0, E[ x0(θθ )2] = (θ θ ) E[xx ](θ θ ) > 0, implyi ng that{x−0} −00 0−00(θ −θ0) = 0 and hence x0θ = x0θ0with positive probabilityunder θ0. Both Φ(v) and Φ(−v) are strictly monotonic,so that x0θ = x0θ0implies both Φ(x0θ) = Φ(x0θ0) andΦ(−x0θ) = Φ(−x0θ0) Therefor e,f(z|θ) = Φ(x0θ)yΦ(−x0θ)1−y= f(z | θ0),Cite as: Victor Chernozhukov, course materials for 14.385 Nonlinear Econometric Analysis, Fall 2007. MIT OpenCourseWare(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].66 66 666with positive probability under θ0.ii) The set Θ = Rp, where p is the dimension of θ, isconvex.iii) By Feller inequality, there is a constant C such thatφ(v)/Φ(v) ≤ C(1 + |v|), so that by integrating there is|2a constant C such that ln Φ(v)| ≤ C(1 + |v| ). ThenE[|ln f(z|θ)|]≤ E[2C(1 + |2x0θ| )] < ∞ by existence ofsecond moments of x.In M-estimation we have θˆ∈ arg i n fˆθ∈ΘQ(θ),with the cr i ter i o n func tio n taking a form of anaverageQˆ(θ) = En[m(zi, θ)].The limit criterion functio n Q(θ) is assumed tobe minimized at the true parameter valu e θ0.Theorem 2 (M-Estimator Consistency): If ziare i.i.d. or stationary and strongly mixing,and ( i ) (Identification) Em(zi, θ) > Em(zi, θ0)for for all θ = θ0; (ii) ( Co mpac tne ss.) Θ isCite as: Victor Chernozhukov, course materials for 14.385 Nonlinear Econometric Analysis, Fall 2007. MIT OpenCourseWare(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].6compact; (i i i ) ( Co n tin u i ty) for each θ, m(z, θ)is continuous at θ with probab i l i ty one; (i v )p(Dominance) E[supθ∈Θ|m(z, θ)|] < ∞; then θˆ→θ0.Proof: The argument is identical to the proofof MLE consistency, apart from verification o fidentifiability, which we assumed here directly.2. Consistency of GMMThe consistenc y result for GMM is similar tothat for MLE. The most important differenceis in the identification hypotheses: h e r e as-sume that¯g(θ) = E[g(z, θ)] = 0must have a uniq u e solution at the true pa-rameter v al u e θ0.Cite as: Victor Chernozhukov, course materials for 14.385 Nonlinear Econometric Analysis, Fall 2007. MIT OpenCourseWare(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].Theorem 3 (GMM Consistency): I f ziare i.i.d.or stationary and strongly mixing (i) E[g(z, θ0)] =p0 and E[g(z, θ)] = 0 for θ = θ0and Aˆ−→ Awith A positive definite; (ii) Θ is compact; ( i i i )for each θ, g(z, θ) is c o n tin u o u s at θ with prob-ability one; (iv)pE[supθ∈Θkg(z, θ)k] < ∞, then θˆ→ θ0.It is often hard to specify primitive conditionsfor the identification condition (i). This con-dition amounts to assuming that there is aunique solution to the set of nonli n e ar