©2004 Royal Statistical Society 1369–7412/04/66735J. R. Statist. Soc. B (2004)66, Part 3, pp. 735–749A method for combining inference across relatednonparametric Bayesian modelsPeter Müller,University of Texas, Houston, USAFernando QuintanaPontificia Universidad Católica de Chile, Santiago, Chileand Gary RosnerUniversity of Texas, Houston, USA[Received March 2003. Final revision January 2004]Summary. We consider the problem of combining inference in related nonparametric Bayesmodels. Analogous to parametric hierarchical models, the hierarchical extension formalizesborrowing strength across the related submodels. In the nonparametric context, modelling iscomplicated by the fact that the random quantities over which we define the hierarchy are infinitedimensional.We discuss a formal definition of such a hierarchical model. The approach includesa regression at the level of the nonparametric model. For the special case of Dirichlet processmixtures, we develop a Markov chain Monte Carlo scheme to allow efficient implementation offull posterior inference in the given model.Keywords: Dependence; Dirichlet process; Hierarchical model; Meta-analysis; Randomfunctions1. IntroductionHierarchical models with nonparametric extensions at various levels of the hierarchy have beendefined and used successfully in the recent literature. MacEachern (1994), Escobar (1994) andEscobar and West (1995) discussed computations in Dirichlet process (DP) mixture modelswhere a parametric prior in a hierarchical model is replaced by the nonparametric DP model.Bush and MacEachern (1996) used a DP prior as random-effects distribution in an analysis-of-variance set-up. Müller and Rosner (1997) used similar DP mixture models to introducenonparametric population distributions for random effects in longitudinal data models. Westet al. (1994) considered normal hierarchical models with DP mixture priors for density esti-mation. Quintana (1998) used hierarchical models with DP priors to assess homogeneity incontingency tables. A recent collection of related review papers can be found in Dey et al. (1998).In this paper we consider an extension of such models to produce combined inference overrelated nonparametric Bayes models, i.e. hierarchical models where each submodel is of non-parametric type. A by-product of this extension is the resulting meta-analysis over models,restricted to the case where the full data sets are available. The approach that we introduceis valid independently of the specific nonparametric model that is chosen for the individualAddress for correspondence: Peter Müller, Department of Biostatistics, M. D. Anderson Cancer Center, Uni-versity of Texas, 1515 Holcombe Boulevard, Box 447, Houston, TX 77030-4009, USA.E-mail: [email protected] P. Müller, F. Quintana and G. Rosnersubmodels. However, the discussion of implementational details and the example are specificto DP mixtures of normals.One solution to achieve combined inference over related nonparametric models is to linkseparate nonparametric models at the level of the hyperparameters only, i.e. independent sub-models conditional on hyperparameters. For example, the base measure in a Dirichlet pro-cess prior for the ith submodel could include a regression on covariates that are specific tothe submodel. This construction is introduced in Cifarelli and Regazzini (1978) as mixturesof products of DPs. The model is used, for example, in Muliere and Petrone (1993). Theydefined dependent nonparametric models for a set of random distributions {Fx, x ∈ X} byassuming marginally for each Fxa DP prior, and introducing a regression in the base mea-sures of these DP priors. Similar models are discussed in Mira and Petrone (1996), Giudiciet al. (2003) and Carota and Parmigiani (2002). Although straightforward, this strategy isstrictly limited to learning about features that can be represented by the hyperparameters. Forexample, consider mixtures of normal submodels where the hyperparameters are the numberof terms in the mixture and mean and variance of a hyperprior on the cluster locations. Ifwe learn in the first study that observations are clustered in a certain way, the only infor-mation that is formally shared with the analysis of the other study is the number of termsand the overall location and variance as represented by the hyperparameters. In other words,learning about specific features of the second study, such as the location of given terms inthe mixture, is not improved by the information that is available from the first study.Tomlinson and Escobar (1999) mitigated this constraint by using a hyperparameter whichitself is a random measure, i.e. a model with a nonparametric hyperprior. MacEachern (1999)discussed an alternative approach for dependent DP models based on introducing correlationsacross the point masses in Sethuraman’s stick breaking construction (Sethuraman, 1994) ofDP models.Many applications that would naturally lead to nonparametric modelling include covari-ates at the level of the nonparametric model. For example, consider a longitudinal model fordrug concentrations over time with a nonparametric prior for patient-specific random effects.It is important that the model incorporates the dependence of the random-effects distribu-tion on known patient-specific covariates, like treatment levels. One approach is discussedin Mallick and Walker (1997) who introduced regression in DP models. They proposed amodel that includes a finite partition of the covariates space, and for each subset of the par-tition they consider a different DP. Of course, this approach only works for finite categoricalcovariates. Alternatively, a straightforward generic strategy for introducing regression in a non-parametric model is to include the covariates in the nonparametric distribution. Consider anonparametric model for an unknown distribution p.θ/, e.g. the random-effects distribution ina longitudinal data model, as mentioned above. To make the model p.θ/ depend on covariatesx, we could consider a joint distribution p.x, θ/. The implied conditional distribution p.θ|x/formalizes the desired density estimation on θ as a function of x. This approach is used, forexample, in Mallet et al. (1988) and Müller and Rosner (1998). However, the approach canbe criticized from a modelling perspective for using the wrong likelihood. Including a jointdistribution p.x, θ/ in the model implies a marginal distribution p.x/. Although x is fixed
View Full Document
Unlocking...