OutlineDistribution of "0362,et-random variablesF-random variablesInference for "0362: t-statisticsWhy reject for large |T|?t vs. NormalConfidence interval for 1Linear combinations of 0, 1A new observation: forecastingGoodness of fitGoodness of fitF test for ``significance'' of regressionWhat can go wrong?Problems in the regression functionProblems with the errors- p. 1/17Statistics 203: Introduction to Regressionand Analysis of VarianceSimple Linear Regression: Inference +DiagnosticsJonathan Taylor●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large|T |?●t vs. Normal●Confidence interval forβ1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 2/17Outline■Inference for vector of coefficients β.■Diagnostics: what can go wrong in our model?●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large|T |?●t vs. Normal●Confidence interval forβ1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 3/17Distribution ofbβ,e■The vectorbβ = (bβ0,bβ1) is a function ofbY so is independentof e.■Bothbβ andbY are linear transformations of Y so they arenormally distributed.■We will proveE((bβ0,bβ1)) = (β0, β1)and has covariance matrixVar(bβ) = σ2n+ σ2X2Sxx−σ2XSxx−σ2XSxxσ2Sxx!■Natural estimates of covariance matrixdVar(bβ) = bσ2n+ bσ2X2Sxx−bσ2XSxx−bσ2XSxxbσ2Sxx!●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large|T |?●t vs. Normal●Confidence interval forβ1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 4/17t-random variables■Start with Z ∼ N(0, 1) is standard normal and G ∼ χ2ν,independent of Z.■ComputeT =ZqGν.■Then T ∼ tνhas a t-distribution with ν degrees of freedom.■Where do they come up in regression?●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large|T |?●t vs. Normal●Confidence interval forβ1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 5/17F -random variables■Start with G1∼ χ2ν1and another independent G2∼ χ2ν2■ComputeF =G1/ν1G2/ν2■Then F ∼ Fν1,ν2has an F -distribution with ν1degrees offreedom in the numerator in ν2in the denominator.■Note: if T ∼ tνthan T2∼ F1,ν.■Where do they come up in regression?●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large |T |?●t vs. Normal●Confidence interval forβ1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 6/17Inference forbβ: t-statistics■Because e is independent ofbβ it follows thatdVar(bβ1) anddVar(bβ0) are independent ofbβ.■Under the hypothesis H0: β1= β01T =bβ1− β01qdVar(bβ1)∼ tn−2.(Why?)■To test this hypothesis, compare |T | to tn−2,1−α/2the 1 − α/2quantile of the t distribution with n − 2 degrees of freedom.■Reject H0if |T | > tn−2,1−α/2.●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large |T |?●t vs. Normal●Confidence interval forβ1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 7/17Why reject for large |T |?■Observing a large |T | is unlikely if β1= β01: reasonable toconclude that H0is false.■Common to report p-valuep − value = 2 ×Z∞|T |ftn−2(s) ds.■Above, ftn−2is the density of a t- random variable with n − 2degrees of freedom.●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large|T |?●t vs. Normal●Confidence interval for β1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 8/17t vs. Normal−3 −2 −1 0 1 2 30.0 0.1 0.2 0.3 0.4sDensity −− f(s)t, 10 dfNormal●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large|T |?●t vs. Normal●Confidence interval for β1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 9/17Confidence interval for β1■For simplicity, writeSE(bβ1) = bσs1Pni=1(Xi− X)2.■Under the model assumptions1 − α = P bβ1− β1SE(bβ1)< tn−2,1−α/2!= Pβ1∈bβ1± tn−2,1−α/2· SE(bβ1)●Outline●Distribution ofbβ,e●t-random variables●F -random variables●Inference forbβ: t-statistics●Why reject for large|T |?●t vs. Normal●Confidence interval forβ1●Linear combinations ofβ0, β1●A new observation:forecasting●Goodness of fit●Goodness of fit●F test for “significance” ofregression●What can go wrong?●Problems in the regressionfunction●Problems with the errors- p. 10/17Linear combinations of β0, β1■It is not too hard to prove that a0bβ0+ a1bβ1is normallydistributed and its standard deviation can be estimated bySE(a0bβ0+ a1bβ1) = bσvuuta20n+(a0X − a1)2Pni=1Xi−