122S:138Bayesian StatisticsBayesian Linear RegressionLecture 15Oct. 22, 2008Kate Cowles374 SH, [email protected] the covariate in(frequentist) simple linear regression• particularly useful when all values of the co-variate are far away from zero and of thesame sign• in this case, without centerin g , intercept isestimated very imprecisely• example: heart rate and body temperaturedata– reponse variable: heart rate in beats/minute– covariate: body temperature in degrees F.– subjects: 130 healthy adults3Intro to Bayesian simple linear regres-sion• likelihoodyi|xi, β0, β1, σ2∼ Nβ0+ β1(xi− ¯x), σ2• “reference prior”: independently uniform onβ0, β1, log σ2p(β0, β1, σ2) ∝1σ2– IG( 0 , 0) on σ2– We approximate this prior in WinBUGSwith∗ vague normals (or “d flat()”) p riors onβ0and β1∗ vague gamma on precision4Analytically computing joint posterior• notationˆβ0= ¯yˆβ1=Pi(xi− ¯x)(yi− ¯y )Pi(xi− ¯x)2SSE =Xi yi−ˆβ0−ˆβ1(xi− ¯x)!2– these are all statistics – fu nctions of thedata alone• joint posterior using reference priorp(β0, β1, σ2|y) ∝1σ21(σ2)(n2)exp−Pi(yi− β0− β1(xi− ¯x))22σ2=1(σ2)(n+22)expSSE + n(β0−ˆβ0)2+P(xi− ¯x)2(β1−ˆβ1)22σ25Steps to find marginal posterior distri-butions• So conditional on σ2p(β0|y, σ2) = N(ˆβ0,σ2n)p(β1|y, σ2) = N(ˆβ1,σ2P(xi− ¯x)2)• if we integrate β1out of the joint posteriorwe getp(β0, σ2|y) ∝1(σ2)(n+12)expSSE + n(β0−ˆβ0)22σ2• now to get margin al of β0, integrate σ2outof the preceding expressionp(β0|y) = t(ˆβ0,s2n, n − 2)a t distribution with meanˆβ0, scales2n, anddegrees of freedom n − 26– recall th at s2=SSEn−2• similarlyp(β1|y) = t(ˆβ1,s2P((xi− ¯x)2, n − 2)• finallyp(σ2)|y = IG(n − 22,SSE2)what GCSR calls a scaled Inverse χ2(n −2, s2)7When will pos teri or be proper with theimproper reference prior?• n > 2• xis n o t all the same8Informative priors in simple linear re-gression• If a previous dataset is available:– See handout from P.M. Lee book for exactmethod if you have only summary statis-tics fro m previous dataset– Or: just combine old and new d atasetsand use reference prior (if yo u have all thedata from old dataset)– Or derive the following simplified , inde-pendent priors (formulas apply if covariatewas centered in previous analysis and youwill center it in your analysis):p(β0) = Nˆβ0,old,s2oldnoldp(β1) = Nˆβ1,old,s2oldPi(xi,ol d− ¯xold)2p(σ2) = IGnold− 22,SSEold29Example• You w ish to use a n article in the literature re-garding a previous study to construct a priorfor a simple linear regression model.• The investigators centered their covariate andreport th e following:n = 100ˆβ0= 5 (s.e.2)ˆβ1= −2 (s.e.1)orˆβ0= 5 95% c.i. (1.04, 8.96)ˆβ1= −2 95% c.i. (−3.98, −0.02)• Recall that:s.e.(ˆβ0) =vuuuuuuts2ns.e.(ˆβ1) =vuuuuuuuts2Pi(xi− ¯x)210s2=SSEn − 2• also:– width of 95% c.i. for β0= 2 tn−2s.e.(ˆβ0)– width of 95% c.i. for β1= 2 tn−2s.e.(ˆβ1)– get t coefficients from t table11Multiple regression• likelihoodyi|xi, β, σ2∼ N (β0+ β1(x1i− ¯x1)+ β(x2i− ¯x)2) + · · · βk−1(xk−1,i− ¯xk−• Reference priorp(β, σ2) ∝1σ2• conditional on σ2, joint posterior of βsp(β|σ2, x, y) = Nˆβ , σ2(XTX)−1• marginally– p(β |x, y) is mul ti vari ate t with n-k de-grees o f freedom– p(σ2|x, y) is IGn−k2,SSE212When is posterior proper with improperreference prior• n > k• columns of X matrix are li nearly indepen-dent13What i f observations yiare not condi-tionally independent, gi ven β, σ2, x?• hierarchical linear models• time series
View Full Document
Unlocking...