Course: Multiple Regression Topic: Non-Linear Associations 1REPRESENTING NON-LINEAR ASSOCIATIONSThus far, we have examined how we can use the OLS regression to test linear associations between variables. A linear relationship between X and Y, for example, implies that the direction of change in Y is constant across the values of X. The following figure displays such a linear association in that the value of Y constantly increases across values of X.However, not all associations are linear in nature. Some associations may be non-linear such that the direction of the relationship between X and Y changes across values of X. The following figures depict 3 possible patterns of non-linear association.The left most figure depicts an inverted quadratic pattern, in which Y increase with increasing values of X until a point at which further increases in X are associated with decreases in Y. The middle figure depicts a cubic pattern in which there are two changes of direction of association between X and & Y across the values of X. The right most figure depicts a quartic pattern in which there are three changes of direction in Y across values of X.Because patterns of association need not be linear in nature, theories may specify non-linear associations. The Yerkes-Dodson Law, for example, suggests that productivity has an inverted quadratic association with physiological arousal, such that increasing levels of arousal increases productivity to a point at which further increases in arousal are detrimental to productivity. Such non-linear associations can be tested with OLS regression.HOW MANY NON-LINEAR ASSOCIATIONS CAN BE TESTEDCertainly, non-linear patterns are not limited to the quadratic, cubic, and quartic patterns displayed above. The direction of association between X and Y may change numerous times across the values of X. Consequently, there are an infinite number of non-linear patterns that can be tested. The number of measured X-values serves as the only limiting factor to the number of patterns that can be tested. For example, we need to have measured at least three values of X to test a quadratic pattern. With only two values of X there is no way to test whether the associationbetween X and Y changes directions across values of X. In general, if A represents the number ofCourse: Multiple Regression Topic: Non-Linear Associations 2distinct values of X that we have measured, we can test A-1 changes in the direction of association between X and Y across values of X. Keep in mind, however, that having sampled “enough” values of X does not imply that a non-linear association will be detected (even if there is a non-linear association in the population). If the sampled values of X do not span the range of X across which the association between X and Y changes direction such a change in direction cannot be detected. For example, imagine that in the population Y increases as X increases to a value of 6 and increases in X beyond 6 lead to decreases in Y. If we sample values of X between 1 and 5, we will detect only a linear increase and the “real” quadratic pattern will be missed. So, an adequate test of a non-linear pattern requires that X be sampled across the range of X at which non-linear patterns occur.Perhaps more important than the question “how many patterns can be tested?” is the question “how many patterns should be tested?” Just because six distinct values of X have been sampled does not mean that a quintic pattern (i.e., 5 direction changes) should be tested. Theory should dictate what patterns should be tested. If a theory dictates only linear patterns, then higherorder trends should best be left alone.SPECIFYING NON-LINEAR ASSOCIATIONS WITH A LINEAR MODELAlthough we are interested in testing non-linear patterns, we will test the non-linear patterns with a linear model. The model is linear in the sense that the model parameters (i.e., betas) are linear. For example, the following model is linear in the parameters.2211XBXBBYoIt is also possible, however, to test non-linear associations with non-linear regression. The following model, for example, is a non-linear model.231221XBXBBYoIn this class, we will focus only on the linear model. The trick to testing non-linear patterns with a linear model is to transform the X-values. That is, parameters remain linear and we incorporate non-linearity by transforming the measured variables. There are two approaches to transforming the X-variable to test for non-linear trends: orthogonal polynomial contrasts and polynomial powers.Orthogonal Polynomial ContrastsWe previously discussed orthogonal polynomial contrasts in the context of ANOVA – so we won’t spend much time with this approach. To use this approach, we transform the X variableby treating it as a nominal variable and weighting each level with the appropriate set of polynomial contrasts. For example, if we measured 4 values of X (i.e., 1, 2, 3, 4) we would treat X as 4-level nominal variable. This 4-level variable would require 3 predictors (i.e., g-1 predictors to represent a G-level nominal variable) in a regression equation. To test the linear, quadratic, and cubic trends, we would create 3 contrast-coded predictors that are weighted with the contrast coefficients that reflect the linear, quadratic, and cubic trends. Keep in mind that those contrasts coefficients are orthogonal and the corresponding predictors will be orthogonal when sample sizes are equal. When sample sizes are not equal, the contrast-coded predictors willCourse: Multiple Regression Topic: Non-Linear Associations 3not be orthogonal and the unique effects of each polynomial contrast will be revealed only when all of the predictors are fully partialled.Power PolynomialsThe other procedure for testing non-linear patterns with a linear model is to transform theX variable using polynomial powers (e.g., X2, X3, X4). Each power represents the number of curves in the regression line (or the number of times Y-changes direction across X). For example,the following regression equation examines the linear and quadratic effects of X on Y.21211XBXBBYoIt is important to note that B2 reflects the quadratic effect of X (i.e. X2) only when the linear effect of X is partialled. Likewise, if we include the cubic effect of X3 the corresponding regression parameter reflects the cubic effect only when
View Full Document
Unlocking...