Markov Chain Monte Carlo Method and Its ApplicationStephen P. BrooksThe Statistician, Vol. 47, No. 1. (1998), pp. 69-100.Stable URL:http://links.jstor.org/sici?sici=0039-0526%281998%2947%3A1%3C69%3AMCMCMA%3E2.0.CO%3B2-0The Statistician is currently published by Royal Statistical Society.Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available athttp://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtainedprior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content inthe JSTOR archive only for your personal, non-commercial use.Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained athttp://www.jstor.org/journals/rss.html.Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.The JSTOR Archive is a trusted digital repository providing for long-term preservation and access to leading academicjournals and scholarly literature from around the world. The Archive is supported by libraries, scholarly societies, publishers,and foundations. It is an initiative of JSTOR, a not-for-profit organization with a mission to help the scholarly community takeadvantage of advances in technology. For more information regarding JSTOR, please contact [email protected]://www.jstor.orgWed Feb 6 14:55:48 2008The Statistician (1 998) 47, Part 1, pp. 69-1 00 Markov chain Monte Carlo method and its application Stephen P. Brooks? University of Bristol, UK [Received April 1997. Revised October 19971 Summary. The Markov chain Monte Carlo (MCMC) method, as a computer-intensive statistical tool, has enjoyed an enormous upsurge in interest over the last few years. This paper provides a simple, comprehensive and tutorial review of some of the most common areas of research in this field. We begin by discussing how MCMC algorithms can be constructed from standard building- blocks to produce Markov chains with the desired stationary distribution. We also motivate and discuss more complex ideas that have been proposed in the literature, such as continuous time and dimension jumping methods. We discuss some implementational issues associated with MCMC methods. We take a look at the arguments for and against multiple replications, consider how long chains should be run for and how to determine suitable starting points. We also take a look at graphical models and how graphical approaches can be used to simplify MCMC implementation. Finally, we present a couple of examples, which we use as case-studies to highlight some of the points made earlier in the text. In particular, we use a simple changepoint model to illustrate how to tackle a typical Bayesian modelling problem via the MCMC method, before using mixture model problems to provide illustrations of good sampler output and of the implementation of a reversible jump MCMC algorithm. Keywords: Bayesian statistics; Gibbs sampler; Metropolis-Hastings updating; Simulation; Software 1. Introduction The integration operation plays a fundamental role in Bayesian statistics. For example, given a sample y from a distribution with likelihood L(y1x) and a prior density for x E R" given by p(x), Bayes's theorem relates the posterior ~(xly)to the prior via the formula where the constant of proportionality is given by Given the posterior, and in the case where x = (xl, x2) is multivariate, for example, we may be interested in the marginal posterior distributions, such as Alternatively, we might be interested in summary inferences in the form of posterior expecta- tions, e.g. ?Addressfor correspondence: Department of Mathematics, University of Bristol, University Walk, Bristol, BS8 lTW, UK. 01998 Royal Statistical Society 0039-0526/98/4706970 S. P. Brooks Thus, the ability to integrate often complex and high dimensional functions is extremely important in Bayesian statistics, whether it is for calculating the normalizing constant in expres- sion (I), the marginal distribution in equation (2) or the expectation in equation (3). Often, an explicit evaluation of these integrals is not possible and traditionally, we would be forced to use numerical integration or analytic approximation techniques; see Smith (1991). However, the Markov chain Monte Carlo (MCMC) method provides an alternative whereby we sample from the posterior directly, and obtain sample estimates of the quantities of interest, thereby performing the integration implicitly. The idea of MCMC sampling was first introduced by Metropolis et al. (1953) as a method for the efficient simulation of the energy levels of atoms in a crystalline structure and was sub-sequently adapted and generalized by Hastings (1970) to focus on statistical problems, such as those described above. The idea is extremely simple. Suppose that we have some distribution n(x), x E E & R", which is known only up to some multiplicative constant. We commonly refer to this as the target distribution. If rr is sufficiently complex that we cannot sample from it directly, an indirect method for obtaining samples from n is to construct an aperiodic and irreducible Markov chain with state space E, and whose stationary (or invariant) distribution is n(x), as discussed in Smith and Roberts (1993), for example. Then, if we run the chain for sufficiently long, simulated values from the chain can be treated as a dependent sample from the target distribution and used as a basis for summarizing important features of z. Under certain regularity conditions, given in Roberts and Smith (1994) for example, the Markov chain sample path mimics a random sample from n.Given realizations {X': t = 0, 1, . . .) from such a chain, typical asymptotic results include the distributional convergence of the realizations, i.e. the distribution of the state of the chain at time t converges to rr as t +m, and the consistency of the ergodic average, i.e., for any scalar functional 0, 5~(XO"lIE,{~(x)} almost surely Many important implementational issues are associated with MCMC methods. These include (among others) the choice of transition mechanism for the chain, the number of