Canonical CorrelationPsy 524Andrew AinsworthMatricesSummaries and reconfigurationTracez sum of diagonal elements 11 12 1321 22 2331 32 3361028724568519aaaBaaaaaaTrace===++=Trace{ If the matrix is an SSCP matrix then the trace is the sum-of-squares{ If the matrix is the variance/covariance matrix than the trace is simply the sum of variances.{ If it is a correlation matrix the trace is just the number of variables.Determinant { this is considered the generalized variance of a matrix. Usually signified by | | (e.g. |A|){ For a 2 X 2 matrix the determinate is simply the product of the main diagonal –the product of the other diagonal11 1221 2211 22 12 21aaDaaDaa aa==−Determinant{ For any other matrices the calculations are best left to computer{ If a determinate of a matrix equals 0 than that matrix cannot inverted, since the inversion process requires division by the determinate. What is a common cause of determinates equaling zero?Eigenvalues and Eigenvectors{ this is a way of rearranging and consolidating the variance in a matrix.1111 MxMMx x MxDV VDany square matrixV EigenvectorEigenvalueλλ====Eigenvalues and Eigenvectors{ Think of it as taking a matrix and allowing it to be represented by a scalar and a vector (actually a few scalars and vectors, because there is usually more than one solution).Eigenvalues and Eigenvectors{ Another way to look at this is:121212()0100010000DIVabcdabcdabcdλνλννλνλνλνλ−=−=−=−=−Eigenvalues and Eigenvectors22If v1 and v2 equal zero the above statement is true, but boring.A non-boring solution comes when the determinate of the leftmost matrix is 0.()() 0() 0Generalize it to 0ad bcad adbcxyzλλλλλλ−−−=−+ + −=−+=2To solve for apply:42yy xyxλλ−± −=Eigenvalues and Eigenvectors2222125142(5 2) 5 * 2 1 * 4 0760774*1*662*1774*1*612*16, 1Dλλλλλλλλ=−++ − =−+=−+ −==−− −====Eigenvalues and Eigenvectors{ Using the first eigenvalue we solve for its corresponding eigenvector12121212156 1042611044This gives you two equations:11 044 011vvvvvvvvV−=−−=−−+ =−==Eigenvalues and Eigenvectors{ Using the second eigenvalue we solve for its corresponding eigenvector12121212251 1042141041This gives you two equations:41 041 014vvvvvvvvV−=−=+=+=−=Eigenvalues and Eigenvectors{ Let’s show that the original equation holds51 1 6 1 6* and 6*42 1 6 1 651 1 1 1 1* and 1*4244 44 ==  −−−−  ==    Canonical CorrelationCanonical Correlation{ measuring the relationship between two separate sets of variables. { This is also considered multivariate multiple regression (MMR)Canonical Correlation{ Often called Set correlationz Set 1 z Set 2 { p doesn’t have to equal q{ Number of cases required ≈ 10 per variable in the social sciences where typical reliability is .80, if higher reliability than less subjects per variable.()1,,pyy…()1,,qxx…Canonical Correlation{ In general, CanCorr is a method that basically does multiple regression on both sides of the equation { this isn’t really what happens but you can think of this way in general.11 2 2 11 22nn nnyy yxx xψψψ βββ++=++Canonical Correlation{ A better way to think about it:z Creating some single variable that represents the Xs and another single variable that represents the Ys.z This could be by merely creating composites (e.g. sum or mean)z Or by creating linear combinations of variables based on shared varianceCanonical CorrelationCanonicalVariate forthe Xsx1x2xnCanonicalVariate forthe Ysy1y2yn{ Make a note that the arrows are coming from the measured variables to the canonical variates.Canonical Correlation{ In multiple regression the linear combinations of Xs we use to predict y is really a single canonical variate.Jargon{ Variables{ Canonical Variates – linear combinations of variablesz One CanV on the X sidez One CanV on the Y side{ Canonical Variate Pair - The two CanVstaken together make up the pair of variatesBackground { Canonical Correlation is one of the most general multivariate forms – multiple regression, discriminate function analysis and MANOVA are all special cases of CanCorr{ Since it is essentially a correlationalmethod it is considered mostly as a descriptive technique.Background{ The number of canonical variate pairs you can have is equal to the number of variables in the smaller set.{ When you have many variables on both sides of the equation you end up with many canonical correlates. Because they are arranged in descending order, in most cases the first couple will be legitimate and the rest just garbage.Questions{ How strongly does a set of variables relate to another set of variables? How strong is the canonical correlation?{ How strongly does a variable relate to its own canonical variate?{ How strongly does a variable relate to the other set’s canonical variate?Assumptions{ Multicollinearity/SingularityCheck Set 1 and Set 2 separatelyz Run correlations and use the collinearity diagnostics function in regular multiple regression{ Outliers – Check for both univariate and multivariate outliers on both set 1 and set 2 separatelyAssumptions{ Normalityz Univariate – univariate normality is not explicitly required for MMRz Multivariate – multivariate normality is required and there is not way to test for except establishing univariate normality on all variables, even though this is still no guarantee.Assumptions{ Linearity – linear relationship assumed for all variables in each set and also between sets{ Homoskedasticity – needs to be checked for all pairs of variables within and between