Principal ComponentsAn Introduction• Exploratory factoring• Meaning & application of “principal components”• Basic steps in a PC analysis• PC extraction process • # PCs determination• PC rotation & interpretation• PC Scores• Tour of PC MAtricesExploratory vs. Confirmatory FactoringExploratory Factoring – when we do not have RH: about . . .• the number of factors • what variables load on which factors• we will “explore” the factor structure of the variables, consider multiple alternative solutions, and arrive at a post hoc solutionWeak Confirmatory Factoring – when we have RH: about the # factors and factor memberships• we will “test” the proposed weak a priori factor structureStrong Confirmatory Factoring – when we have RH: about relative strength of contribution to factors by variables• we will “test” the proposed strong a priori factor structureMeaning of “Principal Components”“Component” analyses are those that are based on the “full” correlation matrix• 1.00s in the diagonal • yep, there’s other kinds, more later“Principal” analyses are those for which each successive factor...• accounts for maximum available variance• is orthogonal(uncorrelated, independent) with all prior factors• full solution (as many factors as variables) accounts for all the varianceComponent ScoresA principal component is a composite variable formed as a linear combination of measure variablesA component SCORE is a person’s score on that composite variable -- when their variable values are applied to the formulas shown belowusually computed from Z-scores of measured variablesthe resulting PC scores are also Z-scores (M=0, S=1)PC1= 11Z1+ 21Z2+ … + k1ZkPC2= 12Z1+ 22Z2+ … + k2Zk (etc.) Component scores have the same properties as the components they represent (e.g., orthogonal or oblique)Proper & Improper Component ScoresA proper component score is a linear combination of all the variables in the analysis the appropriate s applied to variable Z-scoresAn improper component score is a linear combination of the variables which “define”that component usually an additive combination of the Z-scores of the variables with structure weights beyond the chosen cut-off value(Note: improper doesn’t mean “wrong” -- it means “not derived from optimal OLS weightings”)Proper Component ScoresProper component scores are the “instantiation” of the components as they were mathematically derived from R(a linear combination of all the variables)Proper component scores have the same properties as componentsthey are correlated with each other the same as are the PCsPC scores from orthogonal components are orthogonalPC scores from oblique components have r = they can be used to produce the structure matrix (corr of component scores and variables scores), communalities, variance accounted for, etc.Improper Component ScoresImproper component scores are the “instantiation” of the components as they were interpreted by the researcher(a linear combination of the variables which define that component)Improper component scores usually don’t have exactly the same properties as componentsthey are usually correlated with each other whether based on orthogonal or oblique solutionsthey can not be used to produce the structure matrix (corrof component scores and variables scores), communalities, variance accounted for, etc.Tour of PC matricesData matrixXcases,varCorrelation MatrixRvariables, variablesExtraction Structure MatrixSvariables, factorsh2* +* computed by summing squared . weights across a row+ computed by summing squared . . weights down a column^ can only be computed by . . . . . summing across rows if . .. . . . factors are orthogonalFactor Score Coefficient MatrixFSvar, factorFactor Score MatrixFcases, factorsRotated Structure MatrixS’variables, factorsh2^ +Rotated Factor Score Coefficient MatrixFS’var, factorRotated Factor Score MatrixF’cases, factorsTransform-ation MatrixTfactors, factorsRotated Pattern MatrixP’variables, factorsFactor CorrelationMatrixfactors, factorsFor oblique rotations onlyApplications of PC analysisComponents analysis is a kind of “data reduction”• start with an inter-related set of “measured variables”• identify a smaller set of “composite variables” that can be constructed from the “measured variables” and that carry as much of their information as possibleA “Full components solution” ...• has as many PCs as variables• accounts for 100% of the variables’ variance• each variable has a final communality of 1.00 – all of its variance is accounted for by the full set of PCsA “Truncated components solution” …• has fewer PCs than variables• accounts for <100% of the variables’ variance• each variable has a communality < 1.00 -- not all of its variance is accounted for by the PCsThe basic steps of a PC analysis• Compute the correlation matrix• Extract a full components solution• Determine the number of components to “keep”• total variance accounted for• variable communalities• “Rotate” the components and “interpret” (name) them• Structure weights > |.3|-|.4| define which variables “load”• Compute “component scores”• “Apply” components solution• theoretically -- understand meaning of the data reduction• statistically -- use the component scores in other analyses• interpretability• replicabilityPC Factor Extraction• Extraction is the process of forming PCs as linear combinations of the measured variablesPC1= b11X1+ b21X2+ … + bk1XkPC2= b12X1+ b22X2+ … + bk2XkPCf= b1fX1+ b2fX2+ … + bkfXk• Here’s the thing to remember…• We usually perform factor analyses to “find out how many groups of related variables there are” … however …• The mathematical goal of extraction is to “reproduce the variables’ variance, efficiently”PC Factor Extraction, cont.• Consider R on the right• Obviously there are 2 kinds of information among these 4 variables•X1& X2 X3& X4• Looks like the PCs should be formed as,X1X2X3X4X11.0X2.7 1.0X3.3 .3 1.0X4.3 .3 .5 1.0PC1= b11X1+ b21X2+ 0X3+ 0X4PC2= 0X1+ 0X2+ b32X3+ b42X4But remember, PC extraction isn’t trying to “group variables” it is trying to “reproduce variance”• notice that there