Lecture 14: Introduction to Logistic RegressionBinary outcomesExample: Prostate CancerWhat factors are related to capsular penetration?PSAGleason ScoreWhat is Y?Data exploration?What are the problems?Yikes!Properties of the residuals (with linear regression)Clearly, that does not work!“Link” functions: P(Y=1)“Link” functions: YAll have similar propertyAll three togetherSlide 17Focus on Logistic RegressionE(Yi)piFitted values: two typesFitted valuesProstate Cancer ExampleR codeSlide 25Interpreting the outputInferences: Confidence intervalsInferences: Confidence IntervalsConfidence Intervals for ORsProstate ExampleInferences: Hypothesis TestingSlide 32Fitted estimatesFitted values vs. linear predictorEstimationMaximum Likelihood EstimationSlide 37Likelihood Function for “simple” logistic regressionScore functionsData exploration and modelingLogPSASlide 42RcodeModeling, but also model checkingRevised modelSlide 46Model fit?Lecture 14:Introduction to Logistic RegressionBMTRY 701Biostatistical Methods IIBinary outcomesLinear regression is appropriate for continuous outcomesin biomedical research, our outcomes are more commonly of different formsBinary is probably the most prevalent•disease versus not disease•cured versus not cured•progressed versus not progressed•dead versus aliveExample: Prostate Cancer PROSTATE CANCER DATA SETSIZE: 380 observations, 9 variables SOURCE: Hosmer and Lemeshow (2000) Applied Logistic egression: 2nd Edn. 1 Identification Code 1 – 380 ID 2 Tumor Penetration of 0 = No Penetration, CAPSULE Prostatic Capsule 1 = Penetration 3 Age Years AGE 4 Race 1= White, 2 = Black RACE 5 Results of Digital Rectal Exam 1 = No Nodule DPROS 2 = Unilobar Nodule (Left) 3 = Unilobar Nodule (Right) 4 = Bilobar Nodule 6 Detection of Capsular 1 = No, 2 = Yes DCAPS Involvement in Rectal Exam 7 Prostatic Specific Antigen Value mg/ml PSA 8 Tumor Volume from Ultrasound cm3 VOL 9 Total Gleason Score 0 - 10 GLEASONWhat factors are related to capsular penetration?The prostate capsule is the membrane the surrounds the prostate gland As prostate cancer advances, the disease may extend into the capsule (extraprostatic extension) or beyond (extracapsular extension) and into the seminal vesicles. Capsular penetration means a poor prognostic indicator, which accounts for a reduced survival expectancy and a higher progression rate following radical prostatectomy. Let’s start with PSA and Gleason scoreBoth are well-known factors related to disease severityWhat does a linear regression of capsular penetration on PSA and Gleason mean?iieGSPSAY 2`0PSAPSA is the abbreviation for prostate-specific antigen which is an enzyme produced in the epithelial cells of both benign and malignant tissue of the prostate gland. The enzyme keeps ejaculatory fluid from congealing after it has been expelled from the body. Prostate-specific antigen is used as a tumor marker to determine the presence of prostate cancer because a greater prostatic volume, associated with prostate cancer, produces larger amount of prostate-specific antigen. http://www.prostate-cancer.com/Gleason ScoreThe prostate cancer Gleason Score is the sum of the two Gleason grades. After a prostate biopsy, a pathologist examines the samples of prostate cancer cells to see how the patterns, sizes, and shapes are different from healthy prostate cells. Cancerous cells that appear similar from healthy prostate called well-differentiated while cancerous cells that appear very different from healthy prostate cells are called poorly-differentiated. The pathologist assigns one Gleason grade to the most common pattern of prostate cancer cells and then assigns a second Gleason grade to the second-most common pattern of prostate cancer cells. These two Gleason grades indicate prostate cancer’s aggresiveness, which indicates how quickly prostate cancer may extend out of the prostate gland. Gleason score = Gleason 1 + Gleason 2http://www.prostate-cancer.com/What is Y?Y is a binary outcome variableObserved data: •Yi = 1 if patient if patient had capsular involvement•Yi = 0 if patient did not have capsular involvementBut think about the ‘binomial distribution’The parameter we are modeling is a probability, pWe’d like to be able to find a model that relates the probability of capsular involvement to covariatesiieGSPSAYP 2`0)1(For a one-unit increase in GS, we expect the probability of capsularpenetration to increase by β2.Data exploration?0 20 40 60 80 1200.0 0.2 0.4 0.6 0.8 1.0psacap.inv0 2 4 6 80.0 0.2 0.4 0.6 0.8 1.0jitter(gleason)cap.invWhat are the problems?The interpretation does not make sense for a few reasonsYou cannot have P(Y=1) values below 0 or 1What about the behavior of residuals?•normal? •constant variance?Yikes!0 20 40 60 80 120-0.5 0.0 0.5psaregpsa$residuals0 2 4 6 8-1.0 -0.5 0.0 0.5jitter(gleason)(reggs$residuals)Why do they have these strange patterns?(Based on simple linear regressions)Properties of the residuals (with linear regression)Nonnormal error terms•Each error term can only take one of two values:Nonconstant error variance: the variance depends on X:0 1 11010iiiiiiyifxeyifxe)1)(()1()1()ˆ(101022iixxpppppVarClearly, that does not work!A few things to considerWe’d like to model the ‘probability’ of the event occuringY=1 or 0, but we can conceptualize values in between as probabilitiesWe cannot allow probabilities greater than 1 or less than 0“Link” functions: P(Y=1)Logit link:Probit link: Complementary log-log: )1(1)1(log)(logitYPYPY))1(())1((1YPYPprobit))]1(1log(log[))1(log(log  YPYPc“Link” functions: YLogit link:Probit link: Complementary log-log: YYY1log)(logit)()(1YYprobit)]1log(log[)log(log YYc All have similar propertyThey can take any value on the real line for 0 ≤ Y≤ 1Consider logit:•If Y=0, logit(Y) = log(0) = -Inf•If Y=1, logit(Y) = log(Inf) = Inf0.0 0.2 0.4 0.6 0.8 1.0-5 0 5ylog(y/(1 - y))All three together0.0 0.2 0.4 0.6 0.8 1.0-4 -2 0 2 4ylink functionLogitProbitCLogLogAll three together-4 -2 0 2 40.0 0.2 0.4 0.6 0.8 1.0link functionyLogitProbitCLogLogFocus on Logistic RegressionLogistic