ARE 210 Cram r Rao Lower Bound page 1 Econometrics is the application of the tools of mathematical statistics to economic questions and evidence There are many principles and methods that can be applied to estimation and inference in econometrics But all involve optimization of some criterion subject to some constraints 1 BEST minimize the variance of an estimator subject to the constraint that the bias is zero 2 BLUE same as 1 plus a restriction to linear functions of the data the dependent variable 3 MSE minimize the sum of the bias2 and the variance of the estimator 4 Least Squares minimize the sum of squared errors for the estimator 5 Least Absolute Deviations minimize the mean absolute deviation of the estimator from the unknown parameter 6 Maximum Likelihood maximize the joint density function for the data with respect to the parameter estimator 7 Method of Moments use sample moments matched to population moments to obtain robust consistent parameter estimates In some cases several of these procedures coincide For example in the classical linear regression model yt xtT t t i i d n o 2 1 ARE 210 Cram r Rao Lower Bound page 2 the solutions for obtained from 1 2 4 6 and 7 are identical In other situations the estimators differ and can differ substantially in a finite sample We treat the subject matter as the uniform study of the statistical properties of extremum estimators exploiting both general or specific results from the mathematical theory of optimization and from mathematical statistics as each case of interest warrants The Cram r Rao Lower Bound Theorem One of the more important results in classical statistics demonstrates that the principle of a BEST estimator has real content This result is known as the Cram r Rao Lower Bound Theorem which shows that the variance of an unbiased estimator can never fall below a certain well defined lower limit For even the simplest case of a single parameter and an iid vector of observations on a single variable we need some machinery to develop the main ideas involved Let x1 x2 xT be T independent and identically distributed iid observations on a random variable rv x with a probability density function pdf f x where is an unknown parameter to be estimated with the data x1 x2 xT We require the following regularity conditions P1 Define the support set for x by A x f x 0 Then A does not depend upon 2 ARE 210 Cram r Rao Lower Bound 1 Examples f x tion f x P2 page 3 x 1 2 x satisfies P1 but the uniform distribue 2 2 2 1 if 0 x with f x 0 otherwise does not Let X be the sample space for x and let be the parameter space Then for each the derivatives i ln f x i i 1 2 3 exist for all x X P3 ln f x 2 For all 0 E It turns out that these conditions are each essential to the result and also are closely related to other important results on large sample asymptotic properties of estimators The Cram r Rao Theorem establishes an absolute lower bound on the variance of an unbiased estimator for under these conditions Theorem Let g x1 xT be an unbiased estimator for i e E If f x satisfies P1 P3 then var 2 satisfies 2 1 T E ln f x where 2 1 E l 2 1 E l 2 2 l ln L x xT ln f x1 xT T ln f xt t 1 T ln f xt t 1 f x1 xT is the joint pdf for x1 xT and l is the log likelihood function llf for 3 ARE 210 Cram r Rao Lower Bound page 4 Proof First note that identically in the parameter space f x dx 1 implies f x dx 0 and 2 f x 2 dx 0 Therefore E l T f xt T f xt f xt dx1 t 1 t 1 T f t 1 xt f x dx t t f xt dxT 0 since f x dx1 dxt 1dxt 1 t t dxT 1 t 1 T Second because the xt s are iid with pdf f x the random variables ln f xt 4 f xt f xt ARE 210 Cram r Rao Lower Bound page 5 are iid with mean zero Independence implies zero covariances as well f xt f xt E 0 t f x f x t because the joint pdf of xt x is the product of the marginal pdfs f xt x f xt f x t so that f xt f x E f x f x t f xt f x f xt f x f xt x dxt dx f xt f x f xt f x f xt f xt dxt dx f xt f x dx dxt 0 Therefore we have 2 l 2 T ln f xt ln f x T ln f xt E E t 1 t t 1 ln f x 2 T t E t 1 5 ARE 210 Cram r Rao Lower Bound page 6 ln f x 2 T E This implies somewhat obviously now that 1 T E ln f x 2 1 E l 2 establishing the equality of the first two right hand side terms of the CRLB theorem Third the second order partial derivative of l with respect to satisfies 2l 2 2 f x 2 f x 2 t t 2 f xt f xt t 1 T by the quotient rule Upon taking expectations this in turn implies 2 2 2 2 l T f xt f xt E E 2 2 f xt f xt t 1 2 2 f xt 2 T f xt E E f x t t 1 t 1 f xt T T 2 t 1 f xt 2 ln f x 2 t dxt E t 1 T l 2 E 6 ARE 210 since Cram r Rao Lower Bound 2 f xt 2 page 7 dxt 0 This establishes the equality of the second and third right hand side terms of the CRLB theorem We are left with proving the inequality which is straightforward By unbiasedness of we have E g x1 xT f x1 f xT dx1 dxT where we have used the definition of g x1 xT and exploited the iid property of the xt s This equality must hold for all regardless of the true value of the unknown parameter so that differentiation of both sides with respect to gives this is a slight a lg ebraic manipulation 1 T f xt g x1 xT f xt t 1 T t 1 T ln f xt g x1 xT f x1 t 1 f xt dx1 f xT dx1 E Since E 0 as we showed above and E 7 0 dxT dxT ARE 210 Cram r Rao Lower Bound page 8 where is the correlation coefficient of and while 2 2 E we have by the fact that 2 1 2 1 2 2 …