10/15/20081Regression: An IntroductionLIR 832Regression IntroducedTopics of the day:Topics of the day: A. What does OLS do? Why use OLS? How does it work? B. Residuals: What we don’t know. C. Moving to the Multi-variate Model D. Quality of Regression Equations: R210/15/20082Regression Example #1Just what is regression and what can it do?Just what is regression and what can it do? To address this, consider the study of truck driver turnover in the first lecture…10/15/20083Regression Example #2Suppose that we are interested inSuppose that we are interested in understanding the determinants of teacher pay. What we have is a data set on average per-pupil expenditures and average teacher pay by state…10/15/20084Regression Example #2Descriptive Statistics: pay, expendituresVariable N Mean Median TrMean StDev SE Meanpay 51 24356 23382 23999 4179 585expendit 51 3697 3554 3596 1055 148Variable Minimum Maximum Q1 Q3pay 18095 41480 21419 26610expendit 2297 8349 2967 4123Regression Example #2Covariances: pay, expenditurespay expenditpay 17467605expendit 3679754 1112520Correlations: pay, expendituresPearson correlation of pay and expenditures = 0.835P-Value = 0.00010/15/20085Regression Example #2$45,000$$20,000$25,000$30,000$35,000$40,000Avg. Pay$0$5,000$10,000$15,000$0 $1,000 $2,000 $3,000 $4,000 $5,000 $6,000 $7,000 $8,000 $9,000ExpendituresRegression Example #2The regression equation ispay = 12129 + 3.31 expendituresPredictor Coef SE Coef T PConstant 12129 1197 10.13 0.000expendit 3.3076 0.3117 10.61 0.000S = 2325 R-Sq = 69.7% R-Sq(adj) = 69.1%pay = 12129 + 3.31 expenditures is the equation of a line and we can add it to our plot of the data.10/15/20086Regression Example #2$$45,000$15,000$20,000$25,000$30,000$35,000$40,000Avg. PayPay = 12129 +3.31*Expenditures$0$5,000$10,000$0 $1,000 $2,000 $3,000 $4,000 $5,000 $6,000 $7,000 $8,000 $9,000ExpendituresRegression: What Can We Learn?What can we learn from the regression?What can we learn from the regression? Q1: What is the relationship between per pupil expenditures and teacher pay? A: For every additional dollar of expenditure, pay increases by $3.31.10/15/20087Regression: What Can We Learn?Q2:Given our sample is it reasonable toQ2:Given our sample, is it reasonable to suppose that increased teacher expenditures are associated with higher pay? H0: expenditures make no difference: β ≤0 HA: expenditures increase pay: β >0P( (xbarμ)/σ> (3 0370)/ 3117) = p( z > 10 61)P( (xbar -μ)/σ> (3.037 -0)/.3117) = p( z > 10.61) A: Reject our null, reasonable to believe there is a positive relationship.Regression: What Can We Learn?Q3:What proportion of the variance in teacherQ3:What proportion of the variance in teacher pay can we explain with our regression line? A: R-Sq = 69.7%10/15/20088Regression: What Can We Learn?Q4:We can also make predictions from theQ4:We can also make predictions from the regression model. What would teacher pay be if we spent $4,000 per pupil? A: pay = 12129 + 3.31 expenditures pay = 12129 + 3.31*4000 = $25,369 What if we had per pupil expenditures of $6400 (Michigan’s amount)? Pay = 12129 + 3.31*6400 = $33,313Regression: What Can We Learn?Q5:For the states where we have data we can alsoQ5:For the states where we have data, we can also observe the difference between our prediction and the actual amount.  A: Take the case of Alaska: expenditures $8,349actual pay$41,480actual pay$41,480 predicted pay = 12129 + 3.31*8,349 = 38744 difference between actual and predicted pay: 41480 - 38744 = $1,73510/15/20089Regression: What Can We Learn?Note that we have under predicted actual payNote that we have under predicted actual pay. Why might this occur? This is called the residual, it is a measure of the imperfection of our model What is the residual for the state of Maine? per pupil expenditure is $3346 actual teacher pay is $19,583Regression: What Can We Learn?$$45,000Residual (e) = Actual - Predicted$15,000$20,000$25,000$30,000$35,000$40,000Avg. Pay$0$5,000$10,000$0 $1,000 $2,000 $3,000 $4,000 $5,000 $6,000 $7,000 $8,000 $9,000Expenditures10/15/200810Regression NomenclatureSlope CoefficientInterceptY=$0+$1*X+,Dependent VariableExplanatory Variable Residual or Errorpiiii indexes the observationPay=12,129 + 3.31*Expenditure + eSlope CoefficientInterceptiii10/15/200811Components of a Regression ModelDependent variable: we are trying to explain theDependent variable: we are trying to explain the movement of the dependent variable around its mean. Explanatory variable(s): We use these variables to explain the movement of the dependent variable. Error Term: This is the difference between what we can account for with our explanatory variables and th t l l t k b th d d t i blthe actual value taken on by the dependent variable. Parameter: The measure of the relationship between an explanatory variable and a dependent variable.Regression Models are LinearQ: What do we mean by“linear”?Q: What do we mean by linear ? A: The equation takes the form: Ya bXwhereYtheiablebeingpredicted=+:varYtheiablebeingpredictedX the predictor iablea ercept of thelineb slopeof theline:var:var:int:10/15/200812Regression Example #3Using numbers, lets make up an equation for aUsing numbers, lets make up an equation for a compensation bonus system in which everyone starts with a bonus of $500 annually and then receives an additional $100 for every point earned.Bonus Income Job Po s=+$500 $100 * int Now create a table relating job points to bonus incomeRegression Example #310/15/200813Regression Example #3Regression Example #3Basic model takes the form:Basic model takes the form: Y = β0+ β1*X + ε or, for the bonus pay example, Pay = $500 + $100*expenditure + ε10/15/200814Regression Example #3This is the equation of a line where:This is the equation of a line where: $500 is the minimum bonus when the individual has no bonus points. This is the intercept of the line $100 is the increase in the total bonus for every additional job point. This is the