Hierarchical Normal Linear Models WithGroup Specific Intercepts:A Frequentist and Bayesian Perspective22S:138 Bayesian StatisticsLizette OrtegaKristi SwansonMitch ThomannIntroductionThe goal of this analysis is to compare Frequentist and Bayesian linear regression analyses on hierarchical normal linear models with a subject specific intercept. The data will be simulated so that the true values of the regression parameters, as well as the variance components, are known. Then, comparisons of biases of the regression parameters and their interval coverage will be assessed for each method. The Bayesian analysis will be carried out with three different sets of priors to see how the priors affect the parameter estimates. Also, 95%confidence intervals for each parameter will be reported to determine how often the true parameter is contained in the interval. This will be repeated on 25 identically simulated datasets,and average results will be reported. MethodsSimulation:This simulation study used data that was generated using the R statistical package. We generated 25 different datasets simulating repeated measures data using the following method. There were ten different groups that each had five observations. Each of these groups had a subject specific intercept, which was generated as a random normal with mean 0 and variance 100. Each observation also has a random error, which was drawn from a normal with mean 0 and variance 25. There were two predictor variables for each observation, one which was binary and another that was continuous. The response variable was a linear combination of the intercept, five times the continuous variable, negative 25 times the categorical variable, the groupspecific intercept, and the random error.Summary of the simulation:αi ~ N(0,100)eij ~ N(0,25)x1 ~ Uniform(10,30)x2 ~ Bernoulli(.25)yij = 50 + 5x2 – 25x3 + αi + eiji = 1,…,10j = 1,…,5Frequentist Analysis:SAS version 9.2 was used to carry out the Frequentist analysis of the simulated data. TheMIXED procedure was utilized to ascertain parameter estimates, standard errors, and confidence intervals, as well as the covariance matrices for the parameters. Because subjects’ data was generated in such a way to suggest correlation between observations, we specified an unstructured correlation matrix in the procedure. The bias of each parameter estimate and variance component was evaluated by comparing the average value of the estimate to the true value. So the bias results were calculated as follows:Bias(x2) = 5 – ^β1Bias(x3) = - 25 – ^β2The following code is an example of the proc mixed procedure in SAS that was used:proc mixed data = mylib.set1 method=ml;class person;model y1 = x2_1 x3_1 / s covb covbi cl;random person;run;The class statement is used to specify the ten different subjects of the dataset. The model statement specifies the regression model and inputs the observed values for y, x2, and x3. The options listed after the model statement allows retrieval of the fixed effects parameters, the covariance matrix, the inverse covariance matrix, and the confidence limits respectively. Finally,the random statement is used in SAS so that the random group effects are taken in to account andto retrieve estimates for the variance components. Bayesian Analysis:Using R and Winbugs, the 25 dataset were analyzed using a hierarchical normal linear model as follows:yij| α0i, α1i, α2i ~ N(α0i + α1ix1ij + α2ix2ij, σy2)[α0iα1iα2i] | [β0β1β2] , ∑α N3([β0β1β2] , ∑α−1¿[β0β1β2] | [μ0μ1μ2], ∑0 N3([μ0μ1μ2] , ∑0−1¿σy2 ~ Inv-Gamma(α,β)∑α−1 Wishart(R[3,3],6)The parameters that we are primarily interested in this analysis are the regression coefficients, βi. Using Winbugs, we checked that every prior converged for some of the data sets. We found that all of the parameters converged after a burn-in of 20,000 iterations for all of the different priors. Some may have converged before 20,000 iterations, but we chose the largestnumber as the burn-in since we knew it would work for all of the priors. The BGR diagnosticsand auto-correlation plots were used to check for convergence. Below is an example of convergence for the prior from Table 4. All of the priors converged similarly to this.BGR Plots for Results from Table 4 mu.beta[1] chains 1:3iteration20000 22000 24000 0.0 0.5 1.0 1.5mu.beta[2] chains 1:3iteration20000 22000 24000 0.0 0.5 1.0 1.5mu.beta[3] chains 1:3iteration20000 22000 24000 0.0 0.5 1.0 1.5Auto-correlation plots for Table 4mu.beta[1] chains 1:3lag0 20 40 -1.0 -0.5 0.0 0.5 1.0mu.beta[2] chains 1:3lag0 20 40 -1.0 -0.5 0.0 0.5 1.0mu.beta[3] chains 1:3lag0 20 40 -1.0 -0.5 0.0 0.5 1.0ResultsThe following tables contain the Frequentist results.Table 1. Frequentist Regression Parameters and Estimate Bias ResultsParameter Average True ValueAverage Bias MSE Interval CoverageAverage Interval Widthβ049.81086 0.18914 14.55445 96% 19.10194β15.016578 -0.01651 0.01430536 100% 0.54701β2-25.7439 0.74386 3.7889 96% 7.538995σbetween290.23633 9.763672σwithin225.2558 -0.2558These results indicate that the Frequentist analysis performed does a fairly good job of estimating the parameters. The results in Table 1, show that the average bias of both of the regression parameters is fairly close to 0 and the confidence intervals for all parameters has near universal coverage (96% for β0, 100% for β1, 96% for β2).Table 2. Bayesian Regression Parameters and Estimate Bias with prior:[β0β1β2] | [μ0μ1μ2], ∑0 N3(µ, ∑0−1¿, where µ = [000] and ∑0−1¿[1.0−60 00 1.0−600 0 1.0−6]σy2 ~ Inv-Gamma(.001,.001)∑α−1 Wishart(R[3,3],6), where R = [.01 0 00 100 00 0 100]Parameter Average True ValueAverage Bias MSE Interval CoverageAverage Interval Widthβ035.631667 14.368333 222.902892 32% 25.83636β15.677392 -0.67739 0.51135 100% 4.29651636β2-23.76716 -1.23293 17.2062 100% 20.44518Table 3. Bayesian Regression Parameters and Estimate Bias with prior:[β0β1β2] | [μ0μ1μ2], ∑0 N3(µ, ∑0−1¿, where µ = [000] and ∑0−1¿[1.0−60 00 1.0−600 0 1.0−6]σy2 ~ Inv-Gamma(.001,.001)∑α−1 Wishart(R[3,3],6), where R = [100 0 00 .1 00 0 .1]Parameter Average True ValueAverage Bias MSE Interval CoverageAverage IntervalWidthβ050.621129 -0.621129 30.8059225 100% 24.495β14.991935 0.00807 0.07933 100% 1.15123β2-26.995 1.99532 13.9455 100%
View Full Document
Unlocking...