1Section 3.3Canonical Correlation2Objectives Define canonical correlation. Identify the relationship between canonical correlation analysis and multivariate regression. Relate eigenvalues to canonical correlation. Perform canonical variate analysis using PROC REG and PROC CANCORR. Interpret canonical correlation output, test for significance of canonical variates, and save out canonical scores.3Interpreting Multivariate RegressionMultivariateRegressionFull ModelUnivariateRegressionMultivariateRegressionCustom TestsCanonicalCorrelation4The Connection: R2and Eigenvalues Just as HE-1is related to the F-ratio for a univariate model, HT-1is related to R2.  Recall that R2is the square of the multiple correlation between the set of Xs and Y in a linear model. It is computed as SSB/SST. In a multivariate linear model, the eigenvalues of HT-1are a multivariate generalization of the R2 in univariate models.  The eigenvalues of HT-1are the square of the canonical correlation coefficients.5More on the Eigenvalues of HT-1 There are min(p,k) nonzero eigenvalues in multivariate regression. The eigenvalues of HT-1are equal to the square of the canonical correlations. Therefore, the square root of each eigenvalue is a canonical correlation. There are min(p,k) canonical correlations that can be computed for a data set. In the univariate case, min(p,k) = 1, therefore, there is always one value for R2.6The Most General Linear Model CanonicalCorrelationMANOVAMV RegMANCOVAANOVASL REGANCOVACorrelationanalysisT-testsM REG27Canonical Correlation in PROC REGYou can use canonical correlation to understand multivariate multiple regression output. In PROC REG, you can use the CANPRINT option with the MTEST statement. You will see canonical correlations, likelihood ratio significance tests, and information about HT-1 and HE-1.8This demonstration illustrates how to obtain information about canonical correlation in PROC REGCanonical Correlation Using PROC REGch3s3d1.sas9What Have You Learned? Of the two possible dimensions of association between the predictors and the responses, only one of them is statistically significant. One canonical correlation is significantly different from zero, the other is not. 10More Detailed Analysis The REG procedure provides canonical correlation information from a multivariate multiple regression. You may want to investigate the canonical correlations in greater detail. The CANCORR procedure provides a great deal of useful information and statistics for canonical correlation analysis. Canonical correlation may provide more useful and interesting information than multivariate regression analysis.11What Is Really Going On?From variable sets X and Y, find the linear combination of each set of variables that is most highly correlated with a linear combination of the other set. These combinations are the first canonical variates:Repeat this for a second pair of variates, uncorrelated with the first set.111112211122first canonical correlation...... kkppwvwaxax axvbyby byr=+ ++=+ ++=12For Example…Look at the X and Y matrices below:1234 1 2 3Y Y Y Y X X X2358 5233477 4142457 6441346 765 2459 5434366 5254647 5552567 854⎡⎤⎡ ⎤⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎢⎥⎢ ⎥⎣⎦⎣ ⎦313For Example…The best set of coefficients, from canonical correlation analysis: v1and w1make up the first pair of canonical variates. min(p,k), = 3 pairs of variates can be computed for this data set.11112 311234.34 .49 1.22.33 .11 .17 .83.99vwwxx xvyyyyr=−+=−+−=14Canonical Correlation Using PROC CANCORRIn addition to the canonical correlations, you see the following: raw and standardized canonical coefficients for each canonical variate simple correlations for all variable and variate combinations step-down likelihood-ratio tests for canonical correlations multivariate tests for H0:all canonical correlations = 0 adjusted canonical correlations and standard errors of the canonical correlations.15Likelihood-ratio Tests of Canonical VariatesThe likelihood-ratio test of HE-1 is equal to Wilks’ Lambda and can be converted to an approximate F test. This is a step-down method. The null hypothesis is that the canonical correlation Riand all those smaller than Ri= 0. This test is performed sequentially for all canonical variate pairs.16Interpreting the Canonical CorrelationCanonical coefficients are just like regression coefficients: They depend on the other variables in the model. They depend on the scale of measurement. Standardized coefficients address the scaling issue, but they do not address the problem of dependencies among variables.  Coefficients are useful for prediction but not for interpretation.17Interpreting the Canonical CorrelationA more useful way to interpret the canonical correlation in terms of the input variables is to look at the simple correlation statistics. For each pair of variates, look at the correlation between each variable and its canonical variate the correlation between each variable and the canonical variate for the other set of variables.18Assumptions of Canonical Correlation Goal: Description Minimal measurement error Unrestricted variances among variables Similar distributional shapes among variablesGoal: Inference Assumptions for Description, and at least one set of variables follows a multivariate normal distribution419The CANCORR ProcedureGeneral form of the CANCORR procedure: PROC CANCORR <options>;<VAR v-variables;>WITH w-variables;RUN;20Pathological Gambling Example12 Items:DSM-IV Criteria20 Items:Gamblers AnonymousTwo questionnaires:Research questions: Are they measuring the same dimensions? How many dimensions?21This demonstration illustrates the CANCORR procedure for canonical correlation analysisCanonical Correlation Analysis with PROC CANCORRch3s3d2.sas22What Have You Learned? Of the twelve canonical correlations, four are significantly greater than zero.  Each pair of canonical variates has meaningful interpretation. The variables that are most correlated with their own variates are also most correlated with the corresponding variate from the other set of variables.23This exercise reinforces the concepts discussed