Remedial Measures in Regression April 20, 20041Political Science 552Remedial MeasuresOutliers• Discarding Outliers• Truncating Outliers• Robust estimationo LAR/LAD Least Absolute Residuals/Least Absolute Deviations (qreg in Stata)o LMS Least Median Least Squareso IRLS Iteratively Reweighted Least Squares (rreg in Stata)The Heteroscedasticity ProblemiiiXYεββ++=10*iiiuεε=*10 iiiiuXYεββ++=Remedial Measures in Regression April 20, 20042Weighting as a Solution to Heteroscedasticy*10 iiiiuXYεββ++=()*1011iiiiiiuXuYuεββ++=iiiiiiiiuuuXuuY*101εββ++=*101iiiiiiuXuuYεββ++=Variance Proportional to XiikXu =iiiiiiiXXXXXYεββ++=101*101iiiiXXYεββ++=Variance Proportional to Xcontinued*101iiiiXXYεββ++=***1*0*iiiXYεββ++=iiiXYY =*iiXX1*=iiiXεε=*1*0ββ=0*1ββ=Remedial Measures in Regression April 20, 20043Example 1. input Y XY X1. 4 12. 5 13. 4 24. 6 25. 3 36. 5 37. 7 38. 3 49. 6 410. 9 411. endExample 1, continued. regress Y XSource SS df MS Number of obs = 10F( 1, 8) = 0.71Model 2.59173554 1 2.59173554 Prob > F = 0.4224Residual 29.0082645 8 3.62603306 R-squared = 0.0820Adj R-squared = -0.0327Total 31.6 9 3.51111111 Root MSE = 1.9042Y Coef. Std. Err. t P>t [95% Conf. Interval]X .4628099 .5474232 0.85 0.422 -.7995502 1.72517_cons 3.950413 1.595999 2.48 0.038 .2700326 7.630794. predict e,residual. twoway (scatter e X), title(Heteroscedasticity Check)Heteroscedastic ResidualsExample 1-4 -2 0 2 4Residuals1 2 3 4XHeteroscedasticity CheckRemedial Measures in Regression April 20, 20044Example 1, weighted results. * transformation for weighting. gen Ystar=Y/X. gen Xstar=1/X. regress Ystar XstarSource | SS df MS Number of obs = 10-------------+------------------------------ F( 1, 8) = 33.42Model | 12.8065623 1 12.8065623 Prob > F = 0.0004Residual | 3.06565995 8 .383207494 R-squared = 0.8069-------------+------------------------------ Adj R-squared = 0.7827Total | 15.8722223 9 1.76358025 Root MSE = .61904------------------------------------------------------------------------------Ystar | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+----------------------------------------------------------------Xstar | 4.092643 .7079533 5.78 0.000 2.4601 5.725186_cons | .4059945 .389106 1.04 0.327 -.4912856 1.303275------------------------------------------------------------------------------. predict estar,residual. twoway (scatter estar X), title(Heteroscedasticity Check)Untransformed and Transformed Residuals-1 -.5 0 .5 1Residu als1 2 3 4XHeteroscedasticity Check-4 -2 0 2 4Residu als1 2 3 4XHeteroscedasticity CheckOriginal and Transformed DataExample 1| X Y e Ystar Xstar estar | 1. | 1 4 -0.413 4.000 1.000 -0.499 |2. | 1 5 0.587 5.000 1.000 0.501 |3. | 2 4 -0.876 2.000 0.500 -0.452 |4. | 2 6 1.124 3.000 0.500 0.548 |5. | 3 3 -2.339 1.000 0.333 -0.770 |6. | 3 5 -0.339 1.667 0.333 -0.104 |7. | 3 7 1.661 2.333 0.333 0.563 |8. | 4 3 -2.802 0.750 0.250 -0.679 |9. | 4 6 0.198 1.500 0.250 0.071 |10. | 4 9 3.198 2.250 0.250 0.821 |Remedial Measures in Regression April 20, 20045Original and Transformed Equations, Example 10820.0554.463.0596.1950.3ˆ2=+=RXYii8069.0708.903.4389.406.0ˆ2**=+=RXYii0820.0389.406.0708.093.4ˆ2=+=RXYiiWeighted Least Squares Generalized()WyXWXXb′′=−1w()yXXXb′′=−1=×nnnwwwMMMMLL00000021WVariances in Weighted Least Squares{} ( )122−′= WXXbσσw{} ( )122−′= WXXbswws()pnYYwsiiiw−−=∑22ˆRemedial Measures in Regression April 20, 20046Weighted ML()()()∏=−−−−−−−=nipipiiXXYL11,11102221exp2121βββσπσLβ21iiwσ=()()−−−−−=∑∏=−−=nipipiiiniiXXYwwL121,1110121exp221βββπLβGeneralized Least Squares=×nnnnnnnnwwwwwwwwwMMMMLL212222111211WModifying Normal ML for Heteroscedasticity()()∏=−−=niiiYL1222221exp21, βxβσπσσ()()∏=−−=niiiiiYL1222221exp21, βxβσπσσRemedial Measures in Regression April 20, 20047Modeling Heteroscedasticity in ML()()∏=−−=niiiiiYL1222221exp21, βxβσπσσ()kiZZZZg ...,,3212=σ[]ikkiiiZZZZiZZZZeikkiiiγγγγγσγγγγγ+++++==+++++...exp3322110...23322110[]γzγzieii== exp2σModeling Heteroscedasticity in ML continued()()∏=−−=niiiiiYL1222221exp21, βxβσπσσ[]γzγzieii== exp2σ()()∏=−−=niiiYeeLii12221exp21, βxβγzγzπσML Example•ML Modeling variance•Uses routine (hetreg.sta) written by Charles Franklindo d:\courses\ps552\examples\hetreg.staml model lf hetreg (slopes:Y=X)(variance: X)ml max------------------------------------------------------------------------------Y | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------slopes |X | .3854057 .3622433 1.06 0.287 -.3245781 1.09539_cons | 4.122388 .6480185 6.36 0.000 2.852295 5.392481-------------+----------------------------------------------------------------variance | X | 1.043251 .4330151 2.41 0.016 .1945566 1.891945_cons | -2.235108 1.251755 -1.79 0.074 -4.688502 .2182865------------------------------------------------------------------------------Remedial Measures in Regression April 20, 20048ML Estimator ComparediiXY+=554.463.0596.1950.3ˆiiXY+=389.406.0708.093.4ˆOLSWLSiiXY+=362.385.0648.122.4ˆMLVariances under Homoscedasticity()X'XX'1−{} {}IεσXYσ222σ; =={} ( )[]{}( )[]′=−−X'XX'XYσX'XX'bσ1212;{} ( )[]()[]′=−−X'XX'IX'XX'bσ1122σ{} () ()[]()121122−−−== XX'XX'XX'XX'bσσσVariance under Heteroscedasticity{} {}ΩεσXYσ222σ; =={} ( )[]()[]′=−−X'XX'ΩX'XX'bσ1212σ{} ( )[]()[]′=−−X'XX'ΩX'XX'bσ1122σ{} () ()[]1122−−= XX'ΩXX'XX'bσσRemedial Measures in