Regression Estimation January 25, 20041Political Science 552Regression EstimationEstimation AlternativesSum of Square Deviations:Sum of Absolute Deviations:Sum of Deviations:iiXbbY10ˆ+=()∑−=iiYYQˆ()∑−iiYYˆ()2ˆ∑−=iiYYQLeast Squares()∑−−=210 iiXbbYQiiieXbbY ++=10iiiiiYYXbbYeˆ10−=−−=()()∑∑∑−−=−==21022ˆiiiiiXbbYYYeQRegression Estimation January 25, 20042Partial Derivative, b0()∑∑−−==2102iiiXbbYeQ()( )( )()∑∑∑∑∑∑∑++−=++−=−−−=−−−=∂∂iiiiiiiiiiXbnbYXbbYXbbYXbbYbQ2222222120010100adXdadXda22=Partial Derivative, b1()∑∑−−==2102iiiXbbYeQ()( )( )()∑∑∑∑∑∑∑∑++−=++−=−−−=−−−=∂∂20201010122222222iiiiiiiiiiiiiiiiXbXbYXXbXbYXXbbYXXbbYXbQNormal Equations∑∑∑∑∑++−=++−=2101022202220iiiiiiXbXbYXXbnbY()()∑∑−−−=−−−=iiiiiXbbYXXbbY10102020∑∑∑∑∑+=+=21010iiiiiiXbXbYXXbnbYRegression Estimation January 25, 20043Solve for b0∑∑−=iiiXbYnb0nXbnYbii∑∑−=10XbYb10−=∑∑+=iiXbnbY10Solve for b1∑∑∑+=210 iiiiXbXbYX∑∑∑∑∑+−=211 iiiiiiXbXnXbnYYX()∑∑∑∑∑+−=2121 iiiiiiXbnXbnYXYXnXbnYbii∑∑−=10Solve for b1, continued()nYXYXnXbXbiiiiii∑∑∑∑∑−=−2121()nYXYXnXXbiiiiii∑∑∑∑∑−=−221()nXXnYXYXbiiiiii221∑∑∑∑∑−−=Regression Estimation January 25, 20044SSDX()()()()()∑∑∑∑∑∑∑∑∑∑∑∑∑−=+−=+−=+−=+−=−nXXnXnXXnXnXnXXXnXXXXXXXXXiiiiiiiiiiiiii2222222222222222SCP()()()nYXYXnYnXnYnXXnYYXYXnYXXYYXYXXYYXYXYYXXiiiiiiiiiiiiiiiiiiiiii∑∑∑∑∑∑∑∑∑∑∑∑∑∑∑−=+−−=+−−=+−−=−−b1, The Classic Formula()()()∑∑−−−=21XXYYXXbiiiRegression Estimation January 25, 20045X in deviation unitsXXxii−=∑∑∑==21*0iiiixbYxnbYnYxbYnYbiii∑∑===1*0 Normal Equations:Computational ExampleXxYXYxYX2x20-3100-30095 2 70 350 140 25 45 2 60 300 120 25 41 -2 3030-601 42 -1 3570-354 12 -1 2040-204 16 3 70 420 210 36 930401200904 1 50 200 50 16 14 1 55 220 55 16 11 -2 2525-501 433 0 465 1775 380 137 381138017753813703346522========∑∑∑∑∑∑∑nxYXYxXxXYComputation, continued1138017753813703346522========∑∑∑∑∑∑∑nxYXYxXxXY()101133137114653317752221=−×−=−−=∑∑∑∑∑nXXnYXYXbiiiiii3.121133101146510=×−=−=∑∑nXbnYbRegression Estimation January 25, 20046Properties of the Fitted Lineminimum2=∑ie∑= 0ie()∑∑∑∑−=−=iiiiiYYYYeˆˆ∑∑=iiYYˆ()∑∑+=iiXbbY10Properties: Sum of Observed and Fitted Values Are Equal∑∑+=iiXbnbY10ˆ()∑∑+−=iiXbXbYnY11ˆ∑∑∑∑+−=iiiiXbnXnbnYnY11ˆ∑∑∑∑+−=iiiiXbXbYY11ˆ()∑∑+=iiXbbY10ˆProperties: Sum of Product of X’s and e’s Is Zero∑= 0iieXiiiXbbYe10−−=()∑∑−−=iiiiiXbbYXeX10()()∑∑−−−=−−−=iiiiiXbbYXXbbY10102020Regression Estimation January 25, 20047Properties: Sum of Fitted Values and e’s Is Zero()()∑∑−−−=−−−=iiiiiXbbYXXbbY10102020()∑∑−=iiiiiYYYeYˆˆˆ()( )∑∑−−+=iiiiiXbbYXbbeY1010ˆ()()∑∑∑−−+−−=iiiiiiiXbbYXbXbbYbeY101100ˆ∑= 0ˆiieYEstimating σ2{e} (MSE)()2ˆ22}{222−−=−=−==∑∑nYYnenSSEesMSEiiiTSSDrSSE ×−= )1(2()()∑∑∑−=−==nYYYYSSTOSSDiiiT222MLE and Normal Regression()−−=2221exp21σµπσiiYYf()−=−=2222221exp2121exp21iiiYfεσπσσεπσRegression Estimation January 25, 20048ML, continued 1iiiXY10ββε−−=()()()()∏∏=−=−−−=−−−=niiinniiiXYXYL121022212102221021exp221exp21,,ββσπσββσπσσββ()()[]()∑=−−−−−==niiieeeXYnnLLL12102221021021log22log2,,log,,ββσσπσββσββML, Partial Derivatives()∑−−=∂∂iiXYLL10201ββσβ()∑−−=∂∂iiiXYXLL10211ββσβ()21042212∑−−−−=∂∂iiXYnLLββσσσML, Normal