TodaySummary of last classR2 for multiple regressionAdjusted R2Inference in multiple regressionTesting H0:2=0Testing H0:2=0Some detailsOverall goodness of fitDropping subsetsGeneral linear hypothesisAnother fact about multivariate normalPolynomial modelsPolynomial modelsSpline models- p. 1/16Statistics 203: Introduction to Regressionand Analysis of VarianceMultiple Linear Regression: Inference &PolynomialJonathan Taylor●Today●Summary of last class●R2for multiple regression●AdjustedR2●Inference in multipleregression●TestingH0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 2/16Today■Inference: trying to “reduce” model.■Polynomial regression.■Splines + other bases.●Today●Summary of last class●R2for multiple regression●AdjustedR2●Inference in multipleregression●TestingH0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 3/16Summary of last class■Yn×1= Xn×pβp×1+ εn×1■bY = HY, H = X(XtX)−1Xt■e = (I − H)Y■kek2∼ σ2χ2n−p■Generally, if P is a projection onto a subspace˜L such thatP (Xβ) = 0, thenkP Y k2= kP (Xβ + ε)k2= kP εk2∼ σ2χ2dim˜L.●Today●Summary of last class●R2for multiple regression●Adjusted R2●Inference in multipleregression●TestingH0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 4/16R2for multiple regressionSSE =nXi=1(Yi−bYi)2= kY −bY k2SSR =nXi=1(Y −bYi)2= kbY − Y 111k2SST =nXi=1(Yi− Y )2= kY − Y 111k2R2=SSRSST●Today●Summary of last class●R2for multiple regression●Adjusted R2●Inference in multipleregression●TestingH0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 5/16Adjusted R2■As we add more and more variables to the model – evenrandom ones, R2will go to 1.■Adjusted R2tries to take this into account by replacing sumsof squares by “mean” squaresR2a= 1 −SSE/(n − p)SST/(n − 1)= 1 −MSEMST.■Here is an example.●Today●Summary of last class●R2for multiple regression●AdjustedR2●Inference in multipleregression●Testing H0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 6/16Inference in multiple regression■F -statistics.■Dropping a subset of variables.■General linear hypothesis.●Today●Summary of last class●R2for multiple regression●AdjustedR2●Inference in multipleregression●Testing H0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 7/16Testing H0: β2= 0■Can be tested with a t-test:T =bβ2SE(bβ2).■Alternatively, using an F -test with a “full” and “reduced”model◆(F) Yi= β0+ β1Xi1+ β2Xi2+ εi◆(R) Yi= β0+ β1Xi1+ εi■F -statistic: under H0: β2= 0SSEF= kY −bYFk2∼ σ2χ2n−3SSER= kY −bYRk2∼ σ2χ2n−2SSEF− SSER= kbYF−bYRk2∼ σ2χ21and SSEF− SSERis independent of SSEF(see details).●Today●Summary of last class●R2for multiple regression●AdjustedR2●Inference in multipleregression●TestingH0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 8/16Testing H0: β2= 0■Under H0F =(SSEF− SSER)/1SSEF/(n − 3)∼ F1,n−3.■Reject H0at level α if F > F1,n−3,1−α.●Today●Summary of last class●R2for multiple regression●AdjustedR2●Inference in multipleregression●TestingH0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 9/16Some details■SSEF∼ σ2χ2n−3if the full model is correct, andSSER∼ σ2χ2n−2if H0is correct becauseHFY = HF(Xβ + ε) = Xβ + HFεHRY = HR(Xβ + ε) = Xβ + HRε ( under H0)If H0is false SSERis σ2times a non-central χ2n−2.■Why is SSER− SSEFindependent of SSEF?SSER− SSEF= kY − HRY k2− kY − HFY k2= kHRY − HFY k2(Pythagoras)= kHRε − HFεk2( under H0)(HR− HF)ε is in LF, the subspace of the full model whileeF= (I − HF)ε is in L⊥Fthe orthogonal complement of thefull model – therefore eFis independent of (HR− HF)ε.●Today●Summary of last class●R2for multiple regression●AdjustedR2●Inference in multipleregression●TestingH0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 10/16Overall goodness of fit■TestingH0: β1= β2= 0.■Two models:◆(F) Yi= β0+ β1Xi1+ β2Xi2+ εi◆(R) Yi= β0+ εi■F -statistic, under H0:F =(SSER− SSEF)/2SSEF/(n − 3)=k(HR− HF)Y k2/2k(I − HF)Y k2/(n − 3)∼ F2,n−3.■Reject H0if F > F1−α,2,n−3.■Details: same as before.●Today●Summary of last class●R2for multiple regression●AdjustedR2●Inference in multipleregression●TestingH0: β2= 0●Testing H0: β2= 0●Some details●Overall goodness of fit●Dropping subsets●General linear hypothesis●Another fact aboutmultivariate normal●Polynomial models●Polynomial models●Spline models- p. 11/16Dropping subsets■Suppose we have the modelYi= β0+ β1Xi1+ · · · + βp−1Xi,p−1+ εiand we want to test whether we can simplify the model bydropping variables, i.e. testingH0: βj1= · · · = βjk= 0.■Two models:◆(F) – above◆(R) – model with columns Xj1, . . . , Xjkomitted from thedesign matrix.■Under H0F =(SSER− SSEF)/(dfR− dfF)SSEF/dfF∼ FdfR−dfF,dfFwhere dfFand dfRare the “residual” degrees of