Topic2 - Logistic Regression -- JHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 11. Topics........................................... 22. Learning objectives ................................. 33. Inference for logistic regression (LR) models ............. 43.1 LR Model .................................. 43.2 Interpretation of coefficients .................... 63.3 Maximum likelihood estimates and standard errors . . 73.4 Comparing nested models ..................... 83.5 p-value for $j................................ 93.6 Confidence interval for $j..................... 103.7 Standard error for linear combinations of coefficientestimates - FYI ........................... 114. Kyphosis example - Continuous predictor variables ....... 154.1 Variables and questions ...................... 164.2 Display data ............................... 184.3 Model .................................... 224.4 Parameter interpretation ...................... 234.5 Results ................................... 244.6 Likelihood ratio test .......................... 264.7 Correlations among coefficients - FYI ............ 284.8 Several alternative models .................... 314.9 Interpretation............................... 334.10 Checking LR model fit ....................... 344.11 Summary of kyphosis analysis ................ 374.12 Classification using LR: sensitivity, specificity, ROCcurves - FYI.............................. 394.13 Split sample cross-validation - FYI ............. 515. Stata do-file script: cl7ex1.do ....................... 56JHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 21. Topics! Review inference for logistic regression models --estimates, standard errors, confidenceintervals, tests of significance, nested models! Classification using logistic regression: sensitivity,specificity, and ROC curves! Checking the fit of logistic regression models: cross-validation, goodness-of-fit tests, AIC ! Keywords: logistic regression, inference, analysisof deviance, likelihood ratio tests, Wald test,kyphosis, prediction, classification, sensitivity,specificity, ROC curve, cross-validation,Hosmer-Lemeshow statistic, Akaike InformationCriterion (AIC)JHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 32. Learning objectives! Use multiple logistic models to understand howrisk of kyphosis (curvature of the spine)depends on several predictor variables! Use logistic regression to classify subjects andassess the quality of a classification rule with itssensitivity, specificity and ROC curve! Use cross-validation to make unbiasedevaluations of classification rulesJHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 43. Inference for logistic regression (LR) models3.1 LR Model! Recall the LR model:(1) Yi are from a Binomial (ni=1, :i) distribution:i = Pr(Yi=1 | Xis) , n observations(2) Yi are independent(3) log odds(Y=1) =log =3.1 LR Model (cont'd)JHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 5The LR model implies:(a) Odds(Yi=1) = (b) Pr(Yi=1) = =:i = (c) Pr(Yi=0) = 1 - :i = (d) Var(Yi) = :i (1 - :i)JHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 63.2 Interpretation of coefficients! = log odds (Yi=1) , given all Xs = 0 = odds (Yi=1), given all Xs = 0! = Difference in log odds (Yi=1) for Xk+1 -vs- Xk, holding other Xsconstant = odds ratio for Xk +1 -vs- Xk, holding other Xs constantJHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 73.3 Maximum likelihood estimates and standarderrors! The method of maximum likelihood estimationchooses values for parameter estimates whichmake the observed data “maximally likely.” Standard errors are obtained as a by-product ofthe maximization process! Use Stata to get maximum likelihood estimates ( and )and standard errors logit command gives slogistic command gives the sJHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 83.4 Comparing nested models! Null model:log odds (Yi=1) = $0 + $1X1+ þ + $pXp ! Extended model: log odds (Yi=1) = $0 + $1X1 + þ + $pXp + $p+1Xp+1 + þ + $p+sXp+s s “new” Xs! Problem: Test hypothesis that multiple $s = 0: Ho: $p+1 = $p+2 = þ = $p+s = 0! Solution: Use likelihood ratio test (LRT)-2(loglikNULL - loglikEXT) ~ when Ho is trueJHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 93.5 p-value for $j! p-value for H0 vs Ha (two-sided) for any given $jcan be obtained in two ways:(1) Wald test: z = or,(2) Likelihood ratio test (LRT) comparing null(Xj removed) and extended (Xjincluded) models:Likelihood ratio tests are valid under a widerrange of conditions than Wald tests! In Stata, the estimates table gives Wald tests; uselrtest as shown above in the example fornested models to get likelihood ratio testsJHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 103.6 Confidence interval for $j! 100(1-")% CI for $j ± z1-"/2 @ ! In Stata, the estimates table gives CIsJHU Graduate Summer Institute of Epidemiology and Biostatistics, June 16 - June 27, 2003Materials extracted from: Biostatistics 623 © 2002 by JHU Biostatistics Dept.Topic 2 - 113.7 Standard error for linear combinations ofcoefficient estimates - FYI! At times, need to calculate Variance ( )for example, ( w1 = w2 =1 )or,( w1 = 1, w2 = -1 )! Recall formulas for variance calculations— 2