Simple Linear RegressionExampleObjectivesModelAssumptionsEstimationEstimating Model ParametersR ExampleSimple Linear RegressionBret LargetDepartments of Botany and of StatisticsUniversity of Wisconsin—MadisonJanuary 28, 2008Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 1 / 12Simple Linear Regression ExamplePhosphorous ExampleResearchers gathered data to evaluate the use of phosphorus (P) bynine corn plants.The data consist of x, the inorganic P in soil (ppm), and y , theplant-available P (ppm).x 1 4 5 9 13 11 23 23 28y 64 71 54 81 93 76 77 95 109We wish to use the inorganic phosphorous level in the soil to predictthe plant-available phosphorous in the corn plants.It is good practice to put the data into an R data frame.I will show two ways to accomplish this.Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 2 / 12Simple Linear Regression ExampleCreating a Data Frame in R> soilP = c(1, 4, 5, 9, 13, 11, 23, 23, 28)> cornP = c(64, 71, 54, 81, 93, 76, 77, 95, 109)> phos = data.frame(soilP, cornP)> rm(soilP, cornP)> str(phos)'data.frame': 9 obs. of 2 variables:$ soilP: num 1 4 5 9 13 11 23 23 28$ cornP: num 64 71 54 81 93 76 77 95 109Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 3 / 12Simple Linear Regression ExampleCreating a Data Frame in ExcelCreate a spread sheet with a header row with variable names and onerow per observation.Save the file as a comma-separated-variable file (CSV).Read using read.table() with the sep="," argument.> phos2 = read.table("phos.csv", sep = ",", header = T)> str(phos2)'data.frame': 9 obs. of 2 variables:$ soilP: int 1 4 5 9 13 11 23 23 28$ cornP: int 64 71 54 81 93 76 77 95 109Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 4 / 12Simple Linear Regression ExampleGraphical exploration of two quantitative variablessoilPcornP607080901001100 5 10 15 20 25●●●●●●●●●> library(lattice)> plot(xyplot(cornP ~ soilP,+ data = phos, pch = 16))Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 5 / 12Simple Linear Regression ObjectivesObjectives of simple linear regressionDescription To describe the relationship betweeninorganic P in soil and plant-available PEstimation To estimate the population mean plant-available P levelat a given level of inorganic P in soilPrediction To predict the plant-available P level for an individual plantat a given level of inorganic P in soilTesting To test if there is a relationship betweeninorganic P in soil and plant-available PStatistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 6 / 12Simple Linear Regression ModelSimple Linear Regression Modelyi= β0+ β1xi+ ei, ei∼ iid N(0, σ2), i = 1, . . . , ny = β0+ β1x is the “true regression line”β0is the intercept, β1is the slopexiis the explanatory variableyiis the response variableeiis random erroriid stands for independent and identically distributedStatistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 7 / 12Simple Linear Regression AssumptionsSimple Linear Regression Assumptions1The model is correct: E(yi) = β0+ β1xi.2Errors eiare independent.3Errors eihave homogeneous variance: Var(ei) = σ2.4Errors eihave normal distribution: ei∼ N(0, σ2).Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 8 / 12Estimation Estimating Model ParametersEstimating Model ParametersA well estimated line should be “close to the data points”.The least squares criterion says that best line is the one thatminimizesPni=1(yi− (b0+ b1xi))2.The solution to this problem is:ˆβ1=Pni=1(xi− ¯x)(yi− ¯y)Pni=1(xi− ¯x)2ˆβ0= ¯y −ˆβ1¯xThe fitted values are ˆyi=ˆβ0+ˆβ1xiThe estimated variance is ˆσ2=1n−2Pni=1(yi− ˆyi)2Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 9 / 12Estimation Estimating Model ParametersAn Alternative ViewpointThe correlation coefficient r is a number between −1 and 1 thatmeasures the strength of the linear relationship between x and y.r =1n − 1nXi=1xi− ¯xsxyi− ¯ysyThe estimated y for an x that is z standard deviations from the meanis rz standard deviations from the mean.In other words, ˆy = ¯y + rzsy.The estimated slope and intercept are:ˆβ1= rsysx,ˆβ0= ¯y −ˆβ1¯xThe regression line goes through the point (¯x, ¯y).Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 10 / 12R ExampleSimple Linear Regression in R> library(arm)> fit = lm(cornP ~ soilP, data = phos)> display(fit)lm(formula = cornP ~ soilP, data = phos)coef.est coef.se(Intercept) 61.58 6.25soilP 1.42 0.39---n = 9, k = 2residual sd = 10.69, R-Squared = 0.65Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 11 / 12R ExampleSimple Linear Regression in RsoilPcornP607080901001100 5 10 15 20 25●●●●●●●●●> plot(xyplot(cornP ~ soilP,+ data = phos, pch = 16,+ type = c("p", "r")))The argumenttype=c("p","r") tellsxyplot() to plot both pointsand a regression line.Statistics 572 (Spring 2008) Simple Linear Regression January 29, 2008 12 /