Stat 504, Lecture 27 1✬✫✩✪An Introduction toLinear Mixed-Effect ModelsIn Lectures 24 and 26, we looked at the theory andpractice of modeling longitudinal data usinggeneralized estimating equations (GEE). GEEmethods are “semiparametric” because they do notrely on a fully specified probability model. WithGEE, the estimates are efficient if the workingcovariance assumptions are correct. If the workingcovariance assumptions are wrong, the estimatedcoefficients are still approximately unbiased, and SE’sfrom the sandwich (empirical) method are reasonableif the sample is large. The philosophy of GEE is totreat the covariance structure as a nuisance.An alternative to GEE is the class of nonlinearmixed-effect (NLME) models. These are fullyparametric and model the within-subject covariancestructure more explicitly.Stat 504, Lecture 27 2✬✫✩✪NLME’s are best understood as an extension oflinear mixed-effect (LME) models. Therefore,today we’ll cover LME’s even though they were notdesigned for discrete responses. The extension toNLME’s will come next time.Before jumping into LME’s, let’s think briefly aboutclassical ANOVA for repeated measures.Repeated-Measures ANOVA. Suppose thatyi=(yi1,yi2,...,yip)Tdenotes the responses for subject i, i =1,...,n at acommon set of occasions t =(t1,t2,...,tp)T(important!)Assume yi∼ N(µ, Σ) whereµ =(µ1,µ2,...,µp)TThe classical ANOVA decomposition is:Source SS df MSSubjects SSAn − 1 MSAOccasions SSBp − 1 MSBSubj × Occ SSAB(n − 1)(p − 1) MSABStat 504, Lecture 27 3✬✫✩✪To test H0: µ1= µ2= ···= µp, compareF =MSBMSA×Bto an F distribution with (p − 1) and (n − 1)(p − 1)degrees of freedom.This test assumes• balanced data (all subjects measured at alloccasions)• normality• Σ satisfies the Huynh-Feldt circularity conditionOne example of circularity is compound symmetry,which arises whenyij= αi+ µj+ ij,αi∼ N(0,σ2α),ij∼ N(0,σ2),so that V (yij)=σ2α+ σ2andCorr(yij,yij)=σ2τσ2τ+ σ2.Stat 504, Lecture 27 4✬✫✩✪That is, compound symmetry means that any pair ofobservations from the same subject is equallycorrelated, regardless of how far apart in time theyare.• More sophisticated designs (additional within-and between-subjects factors) produce morecomplicated ANOVA tables (Neter, Wassermanand Kutner, ???)• When circularity is violated, we can use moregeneral multivariate regression models (Seber,1985)Stat 504, Lecture 27 5✬✫✩✪Missing data and dropout. Perhaps the mostsevere limitation of classical repeated-measuresANOVA is that it requires the data to be balanced.That is, every subject must be measured at everyoccasion. If one group of subjects is being comparedto another, the group sizes must be equal. In modernlongitudinal studies with human subjects, theseconditions are rarely satisfied. For example, it iscommon for subjects to drop out before the study iscomplete. If a subject drops out before the end of thestudy, it is sensible to use the partial data that he orshe provides; this helps to reduce bias (if dropouts aresystematically different from completers) and makesthe resulting estimates more efficient.For this reason, classical ANOVA is not used veryoften these days; a more common approach is thelinear mixed model, which does not require balance.Stat 504, Lecture 27 6✬✫✩✪Linear mixed modelsAlso known as• multilevel models• linear mixed-effects models• random-effects models• random-coefficient models• hierarchical linear modelsImplemented in• HLM• PROC MIXED• S-PLUS lme()• StataStat 504, Lecture 27 7✬✫✩✪Adopting the notation of Laird and Ware (1982), now“standard” in stat and biostat articles, the model isyi= Xiβ + Zibi+ i,i=1,...,mwhereyi=(yi1,yi2,...,yi,ni)Tbi∼ Nq(0,ψ)i∼ Nni(0,σ2Vi)β = fixed effectsbi= random effects for unit iψ = between-unit covariance matrixσ2Vi= within-unit covariance matrix• Handles unequal ni’s, time-varying covariates,unequally spaced responses• Often we use Vi= I, but other structures—e.g.,autoregressive—are useful, especially when ni’sare large• measurement times are often incorporated intoXi, Zias polynomials• Zicontains a subset of the columns of XiStat 504, Lecture 27 8✬✫✩✪Example. Recall the data from the psychiatric trialin the last lecture.012345602468weeksevPlot of average responses in the placebo and druggroups versus square-root of week:0.0 0.5 1.0 1.5 2.0 2.502468sqrt(week)sevStat 504, Lecture 27 9✬✫✩✪The important covariates are group (0=placebo,1=drug) and√week. Notice that group is acharacteristic of the subject and does not change overtime, whereas the value of√week for any givensubject does change over time. Any covariate whosevalue changes over time for a subject is called“time-varying.” The most common example of atime-varying covariate is time!Suppose we fit a model in which the columns of Xiare• a constant,• group,•√week, and• group ×√weekThis will allow for a different average slope andintercept in each of the two groups. Moreover, if thecolumns of Ziare• a constant and•√week,then the slope and intercept for any given subject willrandomly vary about the group average.Stat 504, Lecture 27 10✬✫✩✪Population-averaged modelAveraging over the distribution of the latent randomeffects bi, the “marginal” (population-averaged)distribution of yiisyi∼ N( Xiβ, Σi)Σi= ZiψZTi+ σ2Vi= σ2W−1iwhere Wi=(ZiξZTi+ Vi)−1and ξ = σ−2ψ• If we take Zi=(1, 1,...,1)T(random intercepts)and Vi= I, then Σ has compound symmetry• If subjects are measured at a small number ofcommon occasions, it becomes possible toestimate a common Σ without assuming anypattern (unstructured) and perform an overallgoodness-of-fit test• Elements of β represent the effects of thecovariates in Xion the mean response, both for asingle subject (i.e., given bi) and on average forthe populationStat 504, Lecture 27 11✬✫✩✪Parameter estimationML estimates of β and the covariance parameters areobtained by maximizing the likelihood functionL ∝σ2−N2 i|Wi|12× expi−12σ2(yi− Xiβ)TWi(yi− Xiβ)For any given covariance parameters, L is maximizedat the GLS estimate˜β =mi=1XTiWiXi−1mi=1XTiWiyiMaximizing a modified version of L (a version with βremoved) produces restricted maximum likelihood(RML) estimates for the covariance parameters• in