WHY YULE-WALKER SHOULD NOT BE USED FORAUTOREGRESSIVE MODELLINGM.J.L. DE HOON, T.H.J.J. VAN DER HAGEN, H. SCHOONEWELLE, AND H. VAN DAMInterfaculty Reactor Institute, Delft University of TechnologyMekelweg 15, 2629 JB Delft, The NetherlandsAbstract - Autoregressive modelling of noise data is widely used for systemidentification, surveillance, malfunctioning detection and diagnosis. Several methods areavailable to estimate an autoregressive model. Usually, the so-called Yule-Walkermethod is employed. The various estimation methods generally yield comparableparameter estimates. In some special cases however, involving nearly periodic signals,the Yule-Walker approach may lead to incorrect parameter estimates. Burg’s methodoffers the best alternative to Yule-Walker. In this paper a theoretical explanation of thisphenomenon is given, while the 1994 IAEA Benchmark test is presented as a practicalexample of Yule-Walker yielding poor parameter estimates.I. INTRODUCTIONAutoregressive modelling of noise data was introduced in nuclear engineering in the mid seventies andgained popularity during the decades thereafter. A historical survey of the gradual acceptation and thediversity of its applications can be found in the so-called SMORN-proceedings (SMORN-III, SMORN-IV,SMORN-V). Nowadays, autoregressive modelling is a widely used means for performing systemidentification, surveillance, malfunctioning detection and diagnosis. Its attractiveness stems, among others,from the fact that the numerical algorithms involved are rather simple.An autoregressive model depends on a limited number of parameters, which are estimated frommeasured noise data. Several methods exist to estimate the autoregressive parameters, such as least-squares, Yule-Walker and Burg’s method. It can be shown that for large data samples these estimationtechniques should lead to approximately the same parameter estimates. Mainly for historical reasons, mostpeople use either the Yule-Walker or the least-squares method. This paper will show, however, that insome special cases the Yule-Walker estimation method leads to poor parameter estimates, even formoderately sized data samples. Least squares should not be used either, as it may lead to an unstablemodel. Burg’s method is preferable.In section II, we will present an overview of the basics of autoregressive modelling. The mathematicalcircumstances causing poor parameter estimates in case of the Yule-Walker technique are described insection III, while some simulations of autoregressive processes are discussed that support our hypotheses.Finally, in section IV, we will illuminate our findings with the application of autoregressive modelling foranomaly detection in the 1994 IAEA Benchmark noise data (Journeau, 1994).II. THEORY OF AUTOREGRESSIVE MODELLINGThe successive samples yt of an autoregressive process linearly depend on their predecessors:y a y a y a yt t t p t p t+ + + + =- - -1 1 2 2Lh, (1)in which ai are the autoregressive parameters and the innovations ht are a stationary purely randomprocess with zero mean. It can be shown that the autocovariance function Rt for delays 0 to p is related tothe autoregressive parameters ai through the Yule-Walker equation for the autoregressive process(Priestley, 1994):R R RR R RR R RaaaRRRppp p p p0 1 11 0 21 2 01212LLM M O MLM M--- -ÊËÁÁÁÁˆ¯˜˜˜˜ÊËÁÁÁÁˆ¯˜˜˜˜= -ÊËÁÁÁÁˆ¯˜˜˜˜. (2)An estimated autoregressive model of the same order p can be written asy a y a y a yt t t p t p t+ + + + =- - -$ $ $$1 1 2 2L h, (3)in which $ai are the autoregressive-parameter estimates and $ht are the estimated innovations. A cleardistinction should be made between the autoregressive process (Eq. (1)) and the correspondingautoregressive model (Eq. (3)) (Broersen and Wensink, 1993). Using Eq. (3), each data sample can bepredicted from its predecessors:$ $y a yt i t iip= --=Â1. (4)As the samples yt cannot be predicted exactly, a residue is introduced, which is defined as the differencebetween the measured value and the estimated value:residue ∫ - =y yt t t$$h, (5)which means that the residue is equal to the estimated innovation, as introduced in Eq. (3).It is assumed in these equations that the autoregressive model order p is known. In practice, the modelorder has to be estimated as well, which is usually done using Akaike’s criterion (Priestley, 1994).Suppose that the estimation realisation y consists of N data points (an estimation realisation containsthose data points that are used for parameter estimation). Three methods of autoregressive-parameterestimation from these data samples shall be considered here, being the least-squares approach (LS), theYule-Walker approach (YW) and Burg’s method (Burg):∑ LS: The total squared residue over the data samples p + 1 to N is minimised, leading to a system oflinear equations:c c cc c cc c caaacccppp p pp p p11 12 121 22 21 21201020LLM M O MLM MÊËÁÁÁÁˆ¯˜˜˜˜ÊËÁÁÁÁˆ¯˜˜˜˜= -ÊËÁÁÁÁˆ¯˜˜˜˜$$$, (6)in which the matrix elementscN py yij t i t jt pN∫-- -= +Â11(7)form an unbiased estimate of the autocovariance function for delay i - j.∑ YW: The first and last p data points are also included in the summation of Eq. (7), resulting in$ $ $$ $ $$ $ $$$$$$$R R RR R RR R RaaaRRRppp ppp0 1 11 0 21 2 01212LLM M O MLMM--- -ÊËÁÁÁÁÁˆ¯˜˜˜˜˜ÊËÁÁÁÁˆ¯˜˜˜˜= -ÊËÁÁÁÁÁˆ¯˜˜˜˜˜, (8)in which the matrix elements $Rt constitute a biased estimate of the autocovariance function (Parzen,1961):$RNy yt ttNt tt∫-= +Â11. (9)The Levinson-Durbin algorithm provides a fast solution of a system of linear equations containing aToeplitz-style matrix as in Eq. (8). Both Eqs. (6) and (8) are in fact approximations to the Yule-Walkerprocess equation (Eq. (2)).∑ Burg: The parameter estimation approach that is nowadays regarded as the most appropriate, is knownas Burg’s method. In contrast to the least-squares and Yule-Walker method, which estimate theautoregressive parameters directly, Burg’s method first estimates the reflection coefficients, which aredefined as the last autoregressive-parameter estimate for each model order p. From these, the parameterestimates are determined using the Levinson-Durbin algorithm. The reflection