Stat 544, Lecture 19 1'&$%Multinomial LogisticRegression ModelsPolytomous responses. Logistic regression can beextended to handle responses that are polytomous,i.e.taking r>2 categories. (Note: The wordpolychotomous is sometimes used, but this word doesnot exist!) When analyzing a polytomous response,it’s important to note whether the response is ordinal(consisting of ordered categories) or nominal(consisting of unordered categories). Some types ofmodels are appropriate only for ordinal responses;other models may be used whether the response isordinal or nominal. If the response is ordinal, we donot necessarily have to take the ordering into account,but it often helps if we do. Using the natural orderingcan• lead to a simpler, more parsimonious model and• increase power to detect relationships with othervariables.Stat 544, Lecture 19 2'&$%If the response variable is polytomous and all thepotential predictors are discrete as well, we coulddescribe the multiway contingency table by aloglinear model. But fitting a loglinear model has twodisadvantages:• It has many more parameters, and many of themare not of interest. The loglinear model describesthe joint distribution of all the variables, whereasthe logistic model describes only the conditionaldistribution of the response given the predictors.• The loglinear model is more complicated tointerpret. In the loglinear model, the effect of apredictor X on the response Y is described bythe XY association. In a logit model, however,the effect of X on Y is a main effect.If you are analyzing a set of categorical variables, andone of them is clearly a “response” while the othersare predictors, I recommend that you use logisticrather than loglinear models.Stat 544, Lecture 19 3'&$%Grouped versus ungrouped. Consider a medicalstudy to investigate the long-term effects of radiationexposure on mortality. The response variable isY =8>>>>><>>>>>:1 if alive,2 if dead from cause other than cancer,3 if dead from cancer other than leukemia,4 if dead from leukemia.The main predictor of interest is level of exposure(low, medium, high). The data could arrive inungrouped form, with one record per subject:low 4med 1med 2high 1...Or it could arrive in grouped form:Exposure Y =1 Y =2 Y =3 Y =4low 22750medium 18 6 7 3high 14 12 9 9Stat 544, Lecture 19 4'&$%In ungrouped form, the response occupies a singlecolumn of the dataset, but in grouped form theresponse occupies r columns. Most computerprograms for polytomous logistic regression canhandle grouped or ungrouped data.Whether the data are grouped or ungrouped, we willimagine the response to be multinomial. That is, the“response” for row i,yi=(yi1,yi2,...,yir)T,is assumed to have a multinomial distribution withindex ni=Prj=1yijand parameterπi=(πi1,πi2,...,πir)T.If the data are grouped, then niis the total number of“trials” in the ith row of the dataset, and yijis thenumber of trials in which outcome j occurred. If thedata are ungrouped, then yihas a 1 in the positioncorresponding to the outcome that occurred and 0’selsewhere, and ni= 1. Note, however, that if the dataare ungrouped, we do not have to actually create adataset with columns of 0’s and 1’s; a single columncontaining the response level 1, 2,...,r is sufficient.Stat 544, Lecture 19 5'&$%Describing polytomous responses by asequence of binary models. In some cases, itmakes sense to “factor” the response into a sequenceof binary choices and model them with a sequence ofordinary logistic models.For example, consider the study of the effects ofradiation exposure on mortality. The four-levelresponse can be modeled in three stages:PopulationAlive DeadNon-cancer CancerOther cancer LeukemiaStage 1Stage 2Stage 3Stat 544, Lecture 19 6'&$%The stage 1 model, which is fit to all subjects,describes the log-odds of death.The stage 2 model, which is fit only to the subjectsthat die, describes the log-odds of death due to cancerversus death from other causes.The stage 3 model, which is fit only to the subjectswho die of cancer, describes the log-odds of death dueto leukemia versus death due to other cancers.Because the multinomial distribution can be factoredinto a sequence of conditional binomials, we can fitthese three logistic models separately. The overalllikelihood function factors into three independentlikelihoods.This approach is attractive when the response can benaturally arranged as a sequence of binary choices.But in situations where arranging such a sequence isunnatural, we should probably fit a singlemultinomial model to the entire response.Stat 544, Lecture 19 7'&$%Baseline-category logit model. Suppose thatyi=(yi1,yi2,...,yir)Thas a multinomial distribution with indexni=Prj=1yijand parameterπi=(πi1,πi2,...,πir)T.When the response categories 1, 2,...,r areunordered, the most popular way to relate πitocovariates is through a set of r − 1 baseline-categorylogits. Taking j∗as the baseline category, the model islog„πijπij∗«= xTiβj,j= j∗.If xihas length p, then this model has (r − 1) × p freeparameters, which we can arrange as a matrix or avector. For example, if the last category is thebaseline (j∗= r), the coefficients areβ =[β1,β2,...,βr−1]Stat 544, Lecture 19 8'&$%orvec(β)=26666664β1β2...βr−137777775.Comments on this model• The kth element of βjcan be interpreted as: theincrease in log-odds of falling into category jversus category j∗resulting from a one-unitincrease in the kth covariate, holding the othercovariates constant.• Removing the kth covariate from the model isequivalent to simultaneously setting j − 1coefficients to zero.• Any of the categories can be chosen to be thebaseline. The model will fit equally well,achieving the same likelihood and producing thesame fitted values. Only the values andinterpretation of the coefficients will change.Stat 544, Lecture 19 9'&$%• To calculate πifrom β, the back-transformation isπij=exp(xTiβj)1+Pk =j∗exp(xTiβk)for the non-baseline categories j = j∗, and thebaseline-category probability isπij∗=11+Pk =j∗exp(xTiβk).Model fitting. This model is not difficult to fit byNewton-Raphson or Fisher scoring. PROCLOGISTIC can do it.Goodness of fit. If the estimated expected countsˆμij= niˆπijare large enough, we can test the fit of ourmodel versus a saturated model that estimates πindependently for i =1,...,N. The deviance forcomparing this model to a saturated one