Today's classModelling correlationOther models of correlationAutoregressive modelsAR(1), =0.95AR(1), =0.5AR(k) modelsAR(2), 1=0.9, 2=-0.2Moving average & ARMA(p,q) modelsARMA(2,4)Stationary time seriesEstimating autocovariance / correlationEstimating power spectrumDiagnostics- p. 1/15Statistics 203: Introduction to Regressionand Analysis of VarianceTime Series: Brief IntroductionJonathan Taylor●Today’s class●Modelling correlation●Other models of correlation●Autoregressive models●AR(1), α = 0.95●AR(1), α = 0.5●AR(k) models●AR(2), α1=0.9, α2= −0.2●Moving average &ARMA(p, q) models●ARMA(2, 4)●Stationary time series●Estimating autocovariance /correlation●Estimating power spectrum●Diagnostics- p. 2/15Today’s class■Models for time-correlated noise.■Stationary time series.■ARMA models.■Autocovariance, power spectrum.■Diagnostics.●Today’s class●Modelling correlation●Other models of correlation●Autoregressive models●AR(1), α = 0.95●AR(1), α = 0.5●AR(k) models●AR(2), α1=0.9, α2= −0.2●Moving average &ARMA(p, q) models●ARMA(2, 4)●Stationary time series●Estimating autocovariance /correlation●Estimating power spectrum●Diagnostics- p. 3/15Modelling correlation■In the mixed effects modelY = Xβ + Zγ + εwith ε ∼ N(0, σ2I) and γ ∼ N(0, D) we were essentiallysayingY ∼ N(Xβ, ZDZt+ σ2I)■We then estimated D from the data (more precisely, R doesthis for us).■We can impose structure on D if necessary. For example, intwo-way random effects ANOVA, we assumed thatαi, βj, (αβ)ijwere independent mean zero normal randomvariables.■In summary, a mixed effect model can be thought of asmodelling the correlation in the errors of Y coming from“sampling from a population.”●Today’s class●Modelling correlation●Other models of correlation●Autoregressive models●AR(1), α = 0.95●AR(1), α = 0.5●AR(k) models●AR(2), α1=0.9, α2= −0.2●Moving average &ARMA(p, q) models●ARMA(2, 4)●Stationary time series●Estimating autocovariance /correlation●Estimating power spectrum●Diagnostics- p. 4/15Other models of correlation■Not all correlations come from sampling.■Another common source is correlation in time.■Example: imagine modelling monthly temperature in a givenlocation over many years.◆Yt= µt%12+ εt, 1 ≤ t ≤ T◆Clearly, µ will vary smoothly as a function of t, but therewill also be correlation in εtdue to “weather systems” thatlast more than one day.◆To estimate µ “optimally” and (especially to) makeinferences about µ we should take these correlations intoaccount.■Time series models are models of such (auto)correlation.Good references: Priestley, “Spectral Theory and TimeSeries”; Brockwell and Davis, “Introduction to Time Seriesand Forecasting.”■Nottingham temperature example.■Today we will just talk about time series in general.●Today’s class●Modelling correlation●Other models of correlation●Autoregressive models●AR(1), α = 0.95●AR(1), α = 0.5●AR(k) models●AR(2), α1=0.9, α2= −0.2●Moving average &ARMA(p, q) models●ARMA(2, 4)●Stationary time series●Estimating autocovariance /correlation●Estimating power spectrum●Diagnostics- p. 5/15Autoregressive models■Simplest stationary “auto”correlationεt= α · εt−1+ ηtwhere η ∼ N(0, σ2I) are i.i.d. Normal random variables,|α| < 1.■This is called an auto-regressive process: “auto” because εis like a regression of ε on its past.■It is called AR(1) because it only goes 1 time point into thepast.■Covariance / correlation functionCov(εt, εt+j) =σ2α|j|1 − α2, Cor(εt, εt+j) = α|j|■Model is “stationary” because Cov(εt, εt+j) depends only on|j|.- p. 6/15AR(1), α = 0.95Timear1.sim10 50 100 150 200−6 −4 −2 0 2 4 6 8- p. 7/15AR(1), α = 0.5Timear1.sim20 50 100 150 200−2 −1 0 1 2●Today’s class●Modelling correlation●Other models of correlation●Autoregressive models●AR(1), α = 0.95●AR(1), α = 0.5●AR(k) models●AR(2), α1=0.9, α2= −0.2●Moving average &ARMA(p, q) models●ARMA(2, 4)●Stationary time series●Estimating autocovariance /correlation●Estimating power spectrum●Diagnostics- p. 8/15AR(k) models■The AR(1) model can be easily generalized to the AR(p)model:εt=pXj=1αjεt−j+ ηtwhere η ∼ N(0, σ2I) are i.i.d. Normal random variables.■Condition on α’s: all roots of the (complex) polynomialφα(z) = 1 −pXj=1αjzjare within the unit disc in the complex plane.- p. 9/15AR(2), α1= 0.9, α2= −0.2Timear2.sim0 50 100 150 200−4 −2 0 2 4●Today’s class●Modelling correlation●Other models of correlation●Autoregressive models●AR(1), α = 0.95●AR(1), α = 0.5●AR(k) models●AR(2), α1=0.9, α2= −0.2●Moving average &ARMA(p, q) models●ARMA(2, 4)●Stationary time series●Estimating autocovariance /correlation●Estimating power spectrum●Diagnostics- p. 10/15Moving average & ARMA(p, q) models■MA(q) is another stationary model:εt=qXj=0βjηt−qwhere η ∼ N(0, σ2I) are i.i.d. Normal random variables.■No conditions on β’s – this is always stationary.■ARMA(p, q) model:εt=pXl=1αlXt−l+qXj=0βjηt−qwhere η ∼ N(0, σ2I) are i.i.d. Normal random variables.- p. 11/15ARMA(2, 4)Timearma.sim0 50 100 150 200−20 0 20 40●Today’s class●Modelling correlation●Other models of correlation●Autoregressive models●AR(1), α = 0.95●AR(1), α = 0.5●AR(k) models●AR(2), α1=0.9, α2= −0.2●Moving average &ARMA(p, q) models●ARMA(2, 4)●Stationary time series●Estimating autocovariance /correlation●Estimating power spectrum●Diagnostics- p. 12/15Stationary time series■In general a (Normally distributed) time series (εt) isstationary ifCov(εt, εt+j) = R(|j|)for some “covariance” function R.■If errors are not normally distributed then the process iscalled weakly stationary, or stationary in mean-square.■The function R(t) can generally be expressed as the Fouriertransform of a spectral densityR(t) =12πZπ−πeitωfR(ω) dωwhere f is called the “spectral” density of the process.■ConverselyfR(t) =Xte−itωR(t)■The function fRis sometimes called the “power spectrum” ofε.●Today’s class●Modelling correlation●Other models of correlation●Autoregressive models●AR(1), α = 0.95●AR(1), α = 0.5●AR(k) models●AR(2), α1=0.9, α2= −0.2●Moving average &ARMA(p, q) models●ARMA(2, 4)●Stationary time