Other Regression ModelsRegression With Categorical PredictorsHandling Categorical PredictorsCategorical Variables ExampleChoosing VariablesA Few Data PointsAnd Regression Says . . .Curvilinear RegressionWhen To Use Curvilinear RegressionTypes of Curvilinear RegressionTransform Them to Linear FormsSample TransformationsSlide 13Corrections to Jain p. 257General TransformationsWhen To Transform?Transforming Due To HomoscedasticityWhat Transformation To Use?Other Tests for TransformationsGeneral Transformation PrincipleExample: Log TransformationConfidence Intervals for Nonlinear RegressionsOutliersDeciding How To Handle OutliersRating vs. AgeCommon Mistakes in RegressionNot Verifying LinearityRelying on Results Without Visual VerificationSome Nonlinear ExamplesAttaching Importance To Values of ParametersNot Specifying Confidence IntervalsNot Calculating Coefficient of DeterminationUsing Coefficient of Correlation ImproperlyUsing Highly Correlated Predictor VariablesUsing Regression Beyond Range of ObservationsUsing Too Many Predictor VariablesMeasuring Too Little of the RangeAssuming Good Predictor Is a Good ControllerWhite SlideOther Regression ModelsAndy WangCIS 5930-03Computer SystemsPerformance Analysis2Regression WithCategorical Predictors•Regression methods discussed so far assume numerical variables•What if some of your variables are categorical in nature?•If all are categorical, use techniques discussed later in the course•Levels - number of values a category can take3HandlingCategorical Predictors•If only two levels, define bi as follows–bi = 0 for first value–bi = 1 for second value•This definition is missing from book in section 15.2•Can use +1 and -1 as values, instead•Need k-1 predictor variables for k levels–To avoid implying order in categories4Categorical Variables Example•Which is a better predictor of a high rating in the movie database, winning an Oscar,winning the Golden Palm at Cannes, or winning the New York Critics Circle?5Choosing Variables•Categories are not mutually exclusive•x1= 1 if Oscar 0 if otherwise•x2= 1 if Golden Palm 0 if otherwise•x3= 1 if Critics Circle Award 0 if otherwise•y = b0+b1 x1+b2 x2+b3 x36A Few Data PointsTitle Rating Oscar Palm NYCGentleman’s Agreement 7.5 X XMutiny on the Bounty 7.6 XMarty 7.4 X X XIf 7.8 XLa Dolce Vita 8.1 XKagemusha 8.2 XThe Defiant Ones 7.5 XReds 6.6 XHigh Noon 8.1 X7And Regression Says . . .• •How good is that?•R2 is 34% of variation–Better than age and length–But still no great shakes•Are regression parameters significant at 90% level?3214.2.1.8.7ˆxxxy 8Curvilinear Regression•Linear regression assumes a linear relationship between predictor and response•What if it isn’t linear?•You need to fit some other type of function to the relationship9When To UseCurvilinear Regression•Easiest to tell by sight •Make a scatter plot–If plot looks non-linear, try curvilinear regression•Or if non-linear relationship is suspected for other reasons•Relationship should be convertible to a linear form10Types ofCurvilinear Regression•Many possible types, based on a variety of relationships:– – – •Many othersbaxy xaby xbay 11Transform Themto Linear Forms•Apply logarithms, multiplication, division, whatever to produce something in linear form•I.e., y = a + b*something•Or a similar form•If predictor appears in more than one transformed predictor variable, correlation likely!12Sample Transformations•For y = aebx, take logarithm of y–ln(y) = ln(a) + bx–y’ = ln(y), b0 = ln(a), b1 = b–Do regression on y’ = b0+b1x•For y = a+b ln(x), –x’ = ex–Do regression on y = a + bln(x’)13Sample Transformations•For y = axb, take log of both x and y–ln(y) = ln(a) + bln(x)–y’ = ln(y), b0 = ln(a), b1 = b, x’ = ex–Do regression on y’ = b0 + b1ln(x’)14Corrections to Jain p. 25715General Transformations•Use some function of response variable y in place of y itself•Curvilinear regression is one example•But techniques are more generally applicable16When To Transform?•If known properties of measured system suggest it•If data’s range covers several orders of magnitude•If homogeneous variance assumption of residuals (homoscedasticity) is violated17Transforming Due To Homoscedasticity•If spread of scatter plot of residual vs. predicted response isn’t homogeneous,•Then residuals are still functions of the predictor variables•Transformation of response may solve the problem18What TransformationTo Use?•Compute standard deviation of residuals at each y_hat–Assume multiple residuals at each predicted value•Plot as function of mean of observations–Assuming multiple experiments for single set of predictor values•Check for linearity: if linear, use a log transform19Other Tests for Transformations•If variance against mean of observations is linear, use square-root transform•If standard deviation against mean squared is linear, use inverse (1/y) transform•If standard deviation against mean to a power is linear, use power transform•More covered in the book20General Transformation PrincipleFor some observed relation between standard deviation and mean,lettransform to and regress on w)(ygs )(yhw  dyygyh)(1)(21Example: Log Transformation•If standard deviation against mean is linear, thenSo  yadyayyh ln1)(yayg )(22Confidence Intervalsfor Nonlinear Regressions•For nonlinear fits using general (e.g., exponential) transformations:–Confidence intervals apply to transformed parameters–Not valid to perform inverse transformation on intervals–Must express confidence intervals in transformed domain23Outliers•Atypical observations might be outliers–Measurements that are not truly characteristic–By chance, several standard deviations out–Or mistakes might have been made in measurement•Which leads to a problem:Do you include outliers in analysis or not?24DecidingHow To Handle Outliers1. Find them (by looking at scatter plot)2. Check carefully for experimental error3. Repeat experiments at predictor values for each outlier4. Decide whether to include or omit outliers–Or do analysis both waysQuestion: Is first point in last lecture’s example an outlier on rating vs. age plot?25Rating vs. Age 6.06.57.07.58.08.59.00 20 40 60 80AgeRating26Common Mistakesin Regression•Generally based on taking shortcuts•Or not being careful•Or not understanding some