Regression with Autocorrelated ErrorsPowerPoint PresentationSlide 3Try this model?Behavior of ResidualsNext step-Slide 7Slide 8Slide 9Looking at the ResidualsHow to proceed?The correct modelRegression + MA diagnosticsSummary – Regression with autocorrelated errorsLong memory processesProperties of a long memory processK. Ensor, STAT 4211Spring 2004Regression with Autocorrelated Errors•Let’s look at the relationship between two U.S. weekly interest rate series measured in percentages– r1(t) = The 1-year Treasury constant maturity rate– r3(t) = The 3-year Treasury constant maturity rate–From 1/5/1962 to 9/10/1999.•Consider the regression model for studying the structure of interest rates r3(tr1(t)+e(t)K. Ensor, STAT 4212Spring 2004Time in weekspercent4 6 8 10 12 14 1601/05/1962 07/18/1969 01/28/1977 08/10/1984 02/21/1992 09/03/1999U.S. Weekly Interest Rates Red line is 1-year; Blue line is 3-yearK. Ensor, STAT 4213Spring 2004yr1yr34 6 8 10 12 14 164 6 8 10 12 14 16Scatterplot of U.S. Weekly Interest Ratediff(yr1)diff(yr3)-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5-1.0 -0.5 0.0 0.5 1.0 1.5Scatterplot of Change 1yr rate and 3yr rateScatterplots between series simultaneous in time, and the change in each series.The two series are highly correlated.K. Ensor, STAT 4214Spring 2004Try this model?Splus summary results of ordinary least squares fit of r3(tr1(t) + e(t)Residual Standard Error = 0.538, Multiple R-Square = 0.9575N = 1967, F-statistic = 44313.8 on 1 and 1965 df, p-value = 0 coef std.err t.stat p.value Inter 0.9107 0.0323 28.2380 0 X 0.9239 0.0044 210.5084 0K. Ensor, STAT 4215Spring 2004Behavior of Residuals0 500 1000 1500 2000-3 -2 -1 0 1 2Time Series Plot of ResidualsLagACF0 5 10 15 20 25 300.0 0.2 0.4 0.6 0.8 1.0 Series : trdiag$std.resThe residuals are nonstionary.K. Ensor, STAT 4216Spring 2004Next step-•From a regression perspective the assumptions of our regression model are violated – long memory / nonstationarity of the residuals.•Let’s consider the change series of interest rates– c1(t)=(1-B)r11(t)– c3(t)=(1-B)r3(t)•Now regress c3 on c1.K. Ensor, STAT 4217Spring 2004Change in 1-year rateTime0 500 1000 1500 2000-1.5 -0.5 0.5 1.5Change in 3-year rate0 500 1000 1500 2000-1.0 0.0 0.5 1.0 1.5K. Ensor, STAT 4218Spring 2004c1c3-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5-1.0 0.0 0.5 1.0 1.5Scatterplot of Change in 1-year and 3-year rate c1 ACF0 5 10 15 20 25 300.0 0.2 0.4 0.6 0.8 1.0 c1 and c30 5 10 15 20 25 300.0 0.2 0.4 0.6 0.8 c3 and c1LagACF-30 -25 -20 -15 -10 -5 00.0 0.2 0.4 0.6 0.8 c3 Lag0 5 10 15 20 25 300.0 0.2 0.4 0.6 0.8 1.0Multivariate Series : cbind(c1, c3)K. Ensor, STAT 4219Spring 2004Regression results OLS fit of c3(t c1(t) + e(t)Residual Standard Error = 0.0682, Multiple R-Square = 0.8479N = 1966, F-statistic = 10947.08 on 1 and 1964 df, p-value = 0 coef std.err t.stat p.value Intercept 0.0002 0.0015 0.1609 0.8722 X 0.7811 0.0075 104.6283 0.0000K. Ensor, STAT 42110Spring 2004Looking at the ResidualsTime0 500 1000 1500 2000-0.4 -0.2 0.0 0.2 0.4Residuals of Change Regression-0.4 -0.2 0.0 0.2 0.40 200 400 600 800HistogramResiduals of Change Regression LagACF0 5 10 15 20 25 30-1.0 -0.5 0.0 0.5 1.0ACFLagACF0 5 10 15 20 25 30-1.0 -0.5 0.0 0.5 1.0PACFThere is a small bit (very small bit) of autocorrelation – violating our regression assumptions.K. Ensor, STAT 42111Spring 2004How to proceed?•The residuals from the regression fit exhibit dependence over the time lags.•Identify the time series model.•Refit Regression + time series model using MLE.K. Ensor, STAT 42112Spring 2004The correct model c3(t)=c1(t) + e(t) with e(t) = a(t) +a(t-1)Parameter estimates: 2) 0.0002 0.7824 0.2115 0.0045Standard errors: ( 0.0018, 0.0077, 0.0221) R-squared = 85.4%K. Ensor, STAT 42113Spring 2004Regression + MA diagnosticsPlot of Standardized Residuals0 500 1000 1500 2000-6 -4 -2 0 2 4 6ACF Plot of ResidualsACF0 10 20 30-1.0 -0.5 0.0 0.5 1.0PACF Plot of ResidualsPACF0 5 10 15 20 25 30-0.10 0.0 0.05P-values of Ljung-Box Chi-Squared StatisticsLagp-value2 4 6 8 10 12 140.0 0.1 0.2 0.3 0.4ARIMA Model Diagnostics: c3ARIMA(0,0,1) Model with Regression Parameters 0.0002408249WE are ignoring theStructure shown in thePartials.K. Ensor, STAT 42114Spring 2004Summary – Regression with autocorrelated errors•Fit regression model•Check residuals for the presence of autocorrelation.–Rather than use the Durbin-Watson statistic commonly recommended in regression texts we use the more general Box-Pierce test. •If autocorrelation is present, identify the nature of the autocorrelation and simultaneously fit (via MLE) the regression parameters and the time series parameters (in Splus use arima.mle)–Note – if your software does not simultaneously fit the parameters and you do not have time to write your own code the interate the fitting of the TS parameters and regression parameters until convergence is reached.K. Ensor, STAT 42115Spring 2004Long memory processes•No memory – ACF is zero•Short memory – stationary–exponential decay in the ACF•Non stationary – random walk–For any fixed lag the ACF will converge to one as the sample size increases to infinity–You will still observe a decay in the sample ACF because we do not have an infinite sample.–How would you design a simulation study to demonstrate this property?•Long memory – stationary–The “differencing” exponent is between –½ and ½.K. Ensor, STAT 42116Spring 2004Properties of a long memory process•If d<.5 – weakly stationary and has an infinite MA average•If d>-.5, then the process can be written as in infinite order AR•The ACF has a nice functional form with the first autocorrelation being d/(1-d). –Note the decay rate – page 73–Polynomial decay rate for d<.5 (not exponential)•The partial acf at lag k is given by d/(k-d)•A process is a fractional ARMA process (ARFIMA) if the fractionally differenced series follows an ARMA(p,q)