The Big PictureMultiple Linear RegressionBret LargetDepartments of Botany and of StatisticsUniversity of Wisconsin—MadisonFebruary 6, 2007Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 1 / 10The Big PictureMultiple Linear RegressionMost interesting questions in biology involve relationships betweenmultiple variables.There are typically multiple explanatory variables.Interactions between variables can be important in understanding aprocess.We will now study statistical models for when there is a singlecontinuous quantitative resp onse variable and multiple explanatoryvariables.Explanatory variables may be quantitative or factors (categoricalvariables).Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 2 / 10The Big PictureModelWe extend simple linear regression to consider models of the followingform:yi= β0+ β1x1i+ β2x2i+ · · · + βkxki+ eiwhere ei∼ iid N(0, σ2) for i = 1, . . . , n.y is the response variable;x1, x2, . . . , xkare the explanatory variables;Some people use the terms dependent and independent variables.I do not like this terminology because the xiare often notindependent.eiare random errors;β0is an intercept and β1, . . . , βkare slopes.Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 3 / 10The Big PictureMultiple Regression ObjectivesInference (estimation and testing) on the model parameters;Estimation/prediction of y at x∗1, x∗2, . . . , x∗kModel selection: Select which explanatory variables are best toinclude in a model.Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 4 / 10The Big PictureEstimation of Regression CoefficientsWe extend the least squares criterion from SLR.Seek the parameters b0, . . . , bkthat minimizenXi=1(yi− (b0+ b1x1i+ b2x2i+ · · · + bkxki))2The solution is the set of estimated coefficientsˆβ0,ˆβ1, . . . ,ˆβk.The ith fitted value is ˆyi=ˆβ0+ˆβ1x1i+ · · · +ˆβkxki.The ith residual is yi− ˆyi.The least square criterion minimizes the sum of the squared residuals,also called the sum of squares for error (SSErr).The estimate of the variance σ2is the mean squared error (MSErr) orˆσ2=Pni=1(yi−ˆyi)2n−(k+1).Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 5 / 10The Big PictureMatrix Notation for EstimatesThere are no simple expressions for the estimated coefficients.The matrix notation solution is concise.y =y1y2...yn, X =1 x11· · · xk11 x12· · · xk2............1 x1n· · · xkn, β =β0β1...βkˆβ =XTX−1XTyˆy = Xˆβ = XXTX−1XTy = HyThe matrix H is called the hat matrix. The diagonal entries are theleverages.Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 6 / 10The Big Picturek = 2 CaseThe model isyi= β0+ β1x1i+ β2x2i+ eiwhere ei∼ iid N(0, σ2) for i = 1, . . . , n.Intercept β0: expected y when x1= 0, x2= 0.Slope β1: expected change in y for 1 unit increase in x1with x2heldconstant.Slope β2: expected change in y for 1 unit increase in x2with x1heldconstant.Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 7 / 10The Big PictureFormulaˆβ1=P(yi− ¯y )(x1i− ¯x1) −[P(yi−¯y)(x2i−¯x2)][P(x1i−¯x1)(x2i−¯x2)]P(x2i−¯x2)2P(x1i− ¯x1)2−[P(x1i−¯x1)(x2i−¯x2)]2P(x2i−¯x2)2ˆβ2=P(yi− ¯y )(x2i− ¯x2) −[P(yi−¯y)(x1i−¯x1)][P(x2i−¯x2)(x1i−¯x1)]P(x1i−¯x1)2P(x2i− ¯x2)2−[P(x2i−¯x2)(x1i−¯x1)]2P(x1i−¯x1)2ˆβ0= ¯y −ˆβ1¯x1−ˆβ2¯x2.Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 8 / 10The Big PicturePesticide ExampleA study was conducted to assess the toxic effect of a pesticide on agiven species of insect.The data consist of:dose rate of the pesticide (x1, units unknown)body weight of an insect (x2, grams, maybe?)rate of toxic action (y , time to death in minutes, maybe?).> toxic = read.table("toxic.txt", header = T)> str(toxic)'data.frame': 19 obs. of 3 variables:$ dose : num 0.696 0.729 0.509 0.559 0.679 0.583 0.742 0.781 0.865 0.723 ...$ weight: num 0.321 0.354 0.134 0.184 0.304 0.208 0.367 0.406 0.49 0.223 ...$ effect: num 0.324 0.367 0.321 0.375 0.345 0.341 0.327 0.256 0.214 0.501 ...Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 9 / 10The Big PictureAnalysisUse R to show graphical analysisUse R to show differences in possible models to fit.> attach(toxic)> fit0 = lm(effect ~ 1)> fit1 = lm(effect ~ dose)> fit2 = lm(effect ~ weight)> fit12 = lm(effect ~ dose + weight)> fit21 = lm(effect ~ weight + dose)Statistics 572 (Spring 2007) Multiple Linear Regression February 8, 2007 10 /
View Full Document
Unlocking...