DOC PREVIEW
UW-Madison STAT 572 - Modeling non-normal data

This preview shows page 1-2-3-4-28-29-30-31-58-59-60-61 out of 61 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 61 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Logistic regression: fitting the modelComponents of generalized linear modelsLogistic regressionCase study: runoff dataCase study: baby foodLogistic regression: InferenceModel fit and model diagnosticsComparing modelsSparse data and the separation problemOutline1Logistic regression: fitting the modelComponents of generalized linear modelsLogistic regressionCase study: runoff dataCase study: baby food2Logistic regression: InferenceModel fit and model diagnosticsComparing modelsSparse data and the separation problemModeling non-normal dataIn all of the linear models we have seen so far, theresponse variable has been modeled with a normaldistribution(response) = (fixed parameters) + (normal error)For many data sets, this model is inadequate.Ex: if the response variable is categorical with two possibleresponses, it makes no sense to model the outcome asnormal.Ex: if the response is always a small positive integer, itsdistribution is also not well described by a normaldistribution.Generalized linear models (GLMs) are an extension oflinear models to model non-normal response variables.Logistic regression is for binary response variables.The link functionStandard linear model:yi= β1xi1+ β2xi2+ · · · + βkxik+ ei, ei∼ N (0, σ2)The mean of expected value of the response is:IE(yi) = β1xi1+ β2xi2+ · · · + βkxikWe will use the notation ηi= β1xi1+ · · · + βkxiktorepresent the linear combination of explanatory variables.In a standard linear model,IE(yi) = ηiIn a GLM, there is a link function g between η and themean of the response variable:g(IE(yi)) = ηiFor standard linear models, the link function is the identityfunction g(yi) = yi.The link functionIt can be easier to consider the inverse of the link function:IE(yi) = g−1(ηi)When the response variable is binary (with values codedas 0 or 1), the mean is simply IEy = IP{y = 1}.A useful function for this case isIEy = IP{y = 1} =eη1 + eη= g−1(η)η can take any value, the mean is always between 0 and 1.The corresponding link function is called the logit function,g(p) = logp1 − p= logIP{Y = 1}IP{Y = 0}It is the log of the odds. Regression under this model iscalled logistic regression.DevianceIn standard linear models, we estimate the parameters byminimizing the sum of the squared residuals.Equivalent to finding parameters that maximize thelikelihood.In a GLM we also fit parameters by maximizing thelikelihood. Thedeviance is negative two times themaximum log likelihood up to an additive constant.Estimation is equivalent to finding parameter values thatminimize the deviance.Logistic regressionLogistic regression is a natural choice when the responseis categorical with two possible outcomes.Pick one outcome to be a “success”, or “yes”, where y = 1.We desire a model to estimate the probability of “success”as a function of the explanatory variables. Using theinverse logit function, the probability of success has theformIP{y = 1} =eη1 + eη=11 + e−ηEquivalent formulas:eη=IP{y = 1}IP{y = 0}η = logIP{Y = 1}IP{Y = 0}We estimate the parameters so that this probability is highfor cases where y = 1 and low for cases where y = 0.Anesthesia exampleIn surgery, it is desirable to give enough anesthetic so thatpatients do not move when an incision is made. It is alsodesirable not to use much more anesthetic than necessary.In an experiment, patients are given differentconcentrations of anesthetic.Response: whether or not they move at the time of incision15 minutes after receiving the drug.Anesthesia dataConcentration0.8 1.0 1.2 1.4 1.6 2.5Move 6 4 2 2 0 0No move 1 1 4 4 4 2Total 7 5 6 6 4 2Proportion 0.17 0.20 0.67 0.67 1.00 1.00Analyze in R with glm twice,once using raw data (0’s and 1’s) andonce using summarized counts (1/7, 1/4, . . . , 4/4, 2/2).Extends chi-square tests.Binomial distributionLogistic regression is related to the binomial distribution.If there are several observations with the same explanatoryvariable values, then the individual responses can beadded up and the sum has a binomial distribution.Recall: the binomial distribution has parameters n and p,mean µ = np and variance σ2= np(1 − p).The probability distribution isIP{X = x} =nxpx(1 − p)n−xLogistic regression is in the “binomial family” of GLMs.Logistic regression in R on raw data> dat = read.table("anesthetic.txt", header = T)> str(dat)’data.frame’: 30 obs. of 3 variables:$ movement: Factor w/ 2 levels "move","noMove": 2 1 2 1 1 ...$ conc : num 1 1.2 1.4 1.4 1.2 2.5 1.6 0.8 1.6 1.4 ...$ nomove : int 1 0 1 0 0 1 1 0 1 0 ...> dat$movement[1] noMove move noMove move move ...[21] ... noMove move noMove move noMoveLevels: move noMove> fit.raw = glm(movement ˜ conc, data=dat, family=binomial)> summary(fit.raw)glm(formula = nomove ˜ conc, family = binomial, data = dat)Estimate Std. Error z value Pr(>|z|)(Intercept) -6.469 2.418 -2.675 0.00748**conc 5.567 2.044 2.724 0.00645**...Null deviance: 41.455 on 29 degrees of freedomResidual deviance: 27.754 on 28 degrees of freedomAIC: 31.754Fitted ModelIP{No move} =eη1 + eη=11 + e−ηwith η = −6.469 + 5.567 × concentrationWe can get predictionsat the ‘link’ level: ηiand at the ‘response’ level: y, or IEY = IP{Y = 1}> predict(fit.raw, type="link")1 2 3 4 5 6 ... 28 29 30-0.90 0.21 1.32 1.32 0.21 7.448 ... 0.21 -0.90 0.21> predict(fit.raw, type="response")1 2 3 4 5 6 ... 28 29 300.29 0.55 0.79 0.79 0.55 0.999 ... 0.55 0.29 0.55Plot of the logit curvelayout(matrix(1:2,2,1))my.etas = seq(-8,8, by=.01)my.prob = 1/(1+exp(-my.etas))plot(my.etas, my.prob, type="l", bty="n",xlab="linear predictor: log-odds eta",ylab="probability of ’success’")abline(h=0); abline(h=1);lines(c(-10,0),c(.5,.5), lty=2)lines(c(0,0),c(0,.5), lty=2)my.conc = seq(0,2.5,by=.05)my.etas = -6.469 + 5.567*my.concmy.prob = 1/(1+exp(-my.etas))plot(my.conc, my.prob, type="l", bty="n", adj=1,xlab="", ylab="prob. no movement")mtext("concentration", side=1, line=0.4)mtext("eta", side=1, line=2.4)mtext("-6.5\n(intercept)",side=1,at=0, line=4)mtext("-0.9\n(-6.5+5.6)",side=1,at=1, line=4)conc.5 = (0-(-6.469))/5.567mtext("0",side=1,at=conc.5, line=3)mtext("4.7\n(-6.5+2*5.6)",side=1,at=2, line=4)lines(c(-1,conc.5),c(.5,.5), lty=2)lines(c(conc.5,conc.5),c(0,.5), lty=2)Plot of movement probability versus concentrationplot(movement ˜ conc, data=dat)plot(movement ˜ as.factor(conc), data=dat)plot(nomove ˜ conc, data=dat)plot(jitter(nomove) ˜ conc, data=dat)plot(jitter(nomove,amount=.02) ˜


View Full Document

UW-Madison STAT 572 - Modeling non-normal data

Download Modeling non-normal data
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Modeling non-normal data and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Modeling non-normal data 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?