Principal Components An IntroductionExploratory vs. Confirmatory FactoringMeaning of “Principal Components”Applications of PC analysisThe basic steps of a PC analysisPC Factor ExtractionPC Factor Extraction, cont.Slide 8Slide 9Slide 10Statistical ProceduresStatistical Procedures, cont.Mathematical ProceduresMathematical Procedure, cont.Slide 15Slide 16Nontrivial Factors ProceduresNontrivial factors Procedures, cont.An Example…Rotation – finding “groups” in the variablesInterpretation – Naming “groups” in the variables“Kinds” of Factors“Kinds” of VariablesPrincipal ComponentsAn Introduction• exploratory factoring• meaning & application of “principal components”• Basic steps in a PC analysis• PC extraction process • # PCs determination• Statistical approaches• Mathematical approaches• “Nontrivial factors” approachesExploratory vs. Confirmatory FactoringExploratory Factoring – when we do not have RH: about . . .• the number of factors • what variables load on which factors• we will “explore” the factor structure of the variables, consider multiple alternative solutions, and arrive at a post hoc solutionWeak Confirmatory Factoring – when we have RH: about the # factors and factor memberships• we will “test” the proposed weak a priori factor structureStrong Confirmatory Factoring – when we have RH: about relative strength of contribution to factors by variables• we will “test” the proposed strong a priori factor structureMeaning of “Principal Components”“Component” analyses are those that are based on the “full” correlation matrix•1.00s in the diagonal •yep, there’s other kinds, more later“Principal” analyses are those for which each successive factor...• accounts for maximum available variance• is orthogonal (uncorrelated, independent) with all prior factors• full solution (as many factors as variables) accounts for all the varianceApplications of PC analysisComponents analysis is a kind of “data reduction”•start with an inter-related set of “measured variables”•identify a smaller set of “composite variables” that can be constructed from the “measured variables” and that carry as much of their information as possibleA “Full components solution” ...•has as many PCs as variables•accounts for 100% of the variables’ variance•each variable has a final communality of 1.00 – all of its variance is accounted for by the full set of PCsA “Truncated components solution” …•has fewer PCs than variables•accounts for <100% of the variables’ variance•each variable has a communality < 1.00 -- not all of its variance is accounted for by the PCsThe basic steps of a PC analysis•Compute the correlation matrix•Extract a full components solution•Determine the number of components to “keep”•total variance accounted for•variable communalities•“Rotate” the components and “interpret” (name) them• Structure weights > |.3|-|.4| define which variables “load”•Compute “component scores” •“Apply” components solution•theoretically -- understand meaning of the data reduction•statistically -- use the component scores in other analyses•interpretability•replicabilityPC Factor Extraction•Extraction is the process of forming PCs as linear combinations of the measured variables PC1 = b11X1 + b21X2 + … + bk1Xk PC2 = b12X1 + b22X2 + … + bk2Xk PCf = b1fX1 + b2fX2 + … + bkfXk•Here’s the thing to remember…•We usually perform factor analyses to “find out how many groups of related variables there are” … however …•The mathematical goal of extraction is to “reproduce the variables’ variance, efficiently”PC Factor Extraction, cont.•Consider R on the right•Obviously there are 2 kinds of information among these 4 variables•X1 & X2 X3 & X4•Looks like the PCs should be formed as, X1 X2 X3 X4X1 1.0X2 .7 1.0X3 .3 .3 1.0X4 .3 .3 .5 1.0 PC1 = b11X1 + b21X2 -- capturing the information in X1 & X2 PC2 = b32X3 + b42X4 -- capturing the information in X3 & X4•But remember, PC extraction isn’t trying to “group variables” it is trying to “reproduce variance”•notice that there are “cross correlations” between the “groups” of variables !!PC Factor Extraction, cont.•So, because of the cross correlations, in order to maximize the variance reproduced, PC1 will be formed more like ...PC1 = .5X1 + .5X2 + .4X3 + .4X4•Notice that all the variables contribute to defining PC1•Notice the slightly higher loadings for X1 & X2•Because PC1 didn’t focus on the X1 & X2 variable group or X3 & X4 variable group, there will still be variance to account for in both, and PC2 will be formed, probably something like …PC2 = .3X1 + .3X2 - .4X3 - .4X4•Notice that all the variables contribute to defining PC2•Notice the slightly higher loadings for X3 & X4PC Factor Extraction, cont.•While this set of PCs will account for lots of the variables’ variance -- it doesn’t provide a very satisfactory interpretation•PC1 has all 4 variables loading on it•PC2 has all 4 variables loading on it and 2 of then have negative weights, even though all the variables are positively correlated with each other•The goal here was point out what extraction does (maximize variance accounted for) and what it doesn’t do (find groups of variables)Determining the Number of PCsDetermining the number of PCs is arguably the most important decision in the analysis …• rotation, interpretation and use of the PCs are all influenced by the how may PCs are “kept” for those processes• there are many different procedures available – none are guaranteed to work !!• probably the best approach to determining the # of PCS…• remember that this is an exploratory factoring -- that meansyou don’t have decent RH: about the number of factors• So … Explore …• consider different “reasonable” # PCs and “try them out” • rotate, interpret &/or tryout resulting factor scores from each and then decide To get started we’ll use the SPSS “standard” of λ > 1.00Statistical Procedures•PC analyses are extracted from a correlation matrix•PCs should only be extracted if there is “systematic covariation” in the correlation matrix•This is know as the “sphericity question”•Note: the test asks if there the next PC should be