Non-linear Components for Multiple RegressionPowerPoint PresentationSlide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Non-linear Components for Multiple Regression•Why we might need non-linear components•Type of non-linear components•Squared-term components & their disadvantages•Centered-squared-term components & their advantages•Quadratic terms, interactions & regression weightsNon-linear Components Although linear regression models do a very good job of capturing or approximating most of the relationships between quantitative variables that psychologists study, there are several notable exceptions.• Study of skills acquisition or other accrual tasks often show ...• initial learning is slow in earlier trials but then speeds up (a) …• learning rates seem to reach a point of diminishing returns (b) …• or both (c) … • depending upon the novelty of the learning task and the number of trials examineda bc“Modeling” these non-linear relationships has two major forms…• actually fitting the mathematical function of the relationship• including nonlinear terms into linear regression• literally asking if there is a linear relationship between the criterion and a non-linear transformation of the predictor• very similar to the process of “trend analysis” in ANOVAAs you might imagine, the latter is more common (and easier)The most common non-linear term (and the only one we’ll cover here) is the quadratic term -- has two forms … U-shaped Inverted-U-shapedThe most common way of including a quadratic term into a linear regression model is to • construct a new variable – a “squared term” that is the square of the original variable X  X2• include the squared term in the model• +b means a U-shaped quadratic component• -b means an inverted-U-shaped quadratic component• determine if the term “adds” to the model •t-test of “b” •F-test of R²-change w/ & w/o squared termFor example, if we wanted to test for a non-linear relationship between practice and performance… compute practsq = pract * pract dep = perform / enter pract paractsqHowever, this common method is not doing quite what we had in mind …The practsq term really combines the linear and the quadratic trends into a single term•a “quadratic only” trend will have the same DV values at each end of the predictor continuum, with a hump in between• however, squaring makes initially larger numbers “more larger” than initially smaller numbers (2*2=4 5*5=25)• so, the squaring process alone will give you a “curved” shape (linear + quadratic), but won’t give you a “humped” or “pure quadratic” shapeThere are two consequences of working with this “combined” term instead of a “quadratic only” term…• the bivariate analysis using this constructed variable won’t be testing what you had intended • the multivariate analysis will have increased collinearity - because both the linear and the “quadratic” term have a linear component Remember that, in the multiple regression, the b for the quadratic term will be “controlled for” the linear term, and so will provide a test of the “pure” quadratic trendIs there a way to get a “pure” quadratic term ???Sure there is…• first center the predictor (subtract the mean of the predictor from each person’s score) X  (X – M)• square this centered score (X – M)2How does that work ???• Consider two persons with scores equally far above and below the mean (mean = 10, p1 = 8 p2 = 12)• After centering, they are equally far from the mean, but with opposite signs (p1 = -2 p2 = 2)• When the centered scores are squared, they are now equally far from the mean & iin the same direction (p1 = p2 = 4)• If there is a “pure” quadratic term, these squared centered scores are now linearly related to the criterionOriginal scores show a “pure” quad trend w/ criterion 8 10 12 -2 0 2Centered scores show the same trend w/ criterionSquared centered scores show a linear trend with criterion 0 4Thus, a quadratic trend between the criterion and the predictor is revealed as a linear trend between the centered & squared predictor and the criterion.But wait – there’s more!!! How do we interpret the quadratic term regression weight ?Some background….To answer that, you have to realize that a quadratic term is “a kind of interaction” – HUH??•the quadratic term tells how the slope of the linear y-x relationship changes as x changes. So, it is a conditional statement… So, it is an interaction!•+b indicates that the y-x linear slope is getting more positiveas x increases•-b indicates that the y-x linear slope is getting less positive (more negative) as x increasesNot sure that makes sense???Take a look at these data… 0 10 20 30 40 50 60Practice% Correct 40 60 80 100There is a definite +linear trend! On average, more practice is better! (a main effect)However, there is also a quadratic trend! How much the next practice will help depends on how much practices you’ve already done. (an interaction)10 practices will help someone who has done 0 practices MORE THAN 10 practices will help someone who has already done 20 !So, how do we interpret the quadratic term regression weight ?Well, there’s a bit of a problem, which can be expressed 2 ways:First, there’s no way to fix the linear term and allow the quadratic term to vary! Said differently, if X changes X2 changes….Second, since it is a kind of interaction term, the change in Y for a 1-unit change in X depends on the starting value of X !!!Here’s an example…We’re going to start with a model that has no linear trend, but a quadratic trend (making it easier to track the quadratic!) Y’ = 0*(X-M) + 2*(X-M)2 + 10So, how do we interpret the quadratic term regression weight ?For X=0  X=1 Y’Δ 10.00  12.00 which is the quadratic coefficient = 2For X=1  X=2 Y’Δ 12.00  18.00 6 ?????We’re going to think about it as an interaction term…In what direction & by how much does the slope of the Y-X regression line as X increases by 1 ???b-quad tells how Y-X slope changes as X changes•As X increases, the Y-X slope becomes more positive•As X decreases, the Y-X slope becomes less positiveb-linear tells Y-X slope when x