Factor AnalysisAssumptionsSlide 3Slide 4Slide 5Slide 6Slide 7Slide 8Extraction MethodsSlide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Rotation MethodsGeometric RotationSlide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Factor AnalysisPsy 524AinsworthAssumptionsAssumes reliable correlationsHighly affected by missing data, outlying cases and truncated dataData screening methods (e.g. transformations, etc.) can greatly improve poor factor analytic resultsAssumptionsSample Size and Missing DataTrue missing data (MCAR) are handled in the usual ways (ch. 4) but regression methods may overfitFactor analysis needs large samples and it is one of the only draw backs•The more reliable the correlations are the smaller the number of subjects needed•Need enough subjects for stable estimatesAssumptionsComrey and Lee•50 very poor, 100 poor, 200 fair, 300 good, 500 very good and 1000+ excellent •Shoot for minimum of 300 usually•More highly correlated markers less subjectsAssumptionsNormalityUnivariate - normally distributed variables make the solution stronger but not necessaryMultivariate – is assumed when assessing number of factors; usually tested univariatelyAssumptionsNo outliers – obvious influence on correlations would bias resultsMulticollinearity/SingularityIn PCA it is not problem; no inversionsIn FA, if det(R) or any eigenvalue approaches 0 -> multicollinearity is likelyAlso investigate inter-item SMCs approaching 1AssumptionsFactorable R matrixNeed inter-item correlations > .30 or FA is unlikelyLarge inter-item correlations does not guarantee solution either•Duos•MultidimensionalityMatrix of partials adjusted for other variablesOther testsAssumptionsVariables as outliersSome variables don’t workExplain very little varianceRelates poorly with factorLow SMCs with other itemsLow loadingsExtraction MethodsThere are many (dozens at least)All extract orthoganal sets of factors (components) that reproduce the R matrixDifferent techniques – some maximize variance, others minimize the residual matrix (R – reproduced R)With large stable sample they all should be relatively the sameExtraction MethodsUsually un-interpretable without rotation (next)Differ in output depending on combinations ofExtraction methodCommunality estimatesNumber of factors extractedRotational MethodExtraction MethodsPCA vs. FA (family)PCA begins with 1s in the diagonal of the correlation matrix; all variance extracted; each variable giving equal weight; outputs inflated communality estimateFA begins with a communality estimates (e.g. SMC) in the diagonal; analyzes only common variance; outputs a more realistic communality estimateExtraction MethodsPCA analyzes varianceFA analyzes covariance (communality)PCA reproduces the R matrix (near) perfectlyFA is a close approximation to the R matrixExtraction MethodsPCA – the goal is to extract as much variance with the least amount of factorsFA – the goal is to explain as much of the correlations with a minimum number of factorsPCA gives a unique solutionFA can give multiple solutions depending on the method and the estimates of communalityExtraction MethodsPCAExtracts maximum variance with each componentFirst component is a linear combination of variables that maximizes component score variance for the casesThe second (etc.) extracts the max. variance from the residual matrix left over after extracting the first component (therefore orthogonal to the first)If all components retained, all variance explainedExtraction MethodsPrincipal (Axis) FactorsEstimates of communalities (SMC) are in the diagonal; used as starting values for the communality estimation (iterative)Removes unique and error varianceSolution depends on quality of the initial communality estimatesExtraction MethodsMaximum LikelihoodComputationally intensive method for estimating loadings that maximize the likelihood (probability) of the correlation matrix.Unweighted least squares – ignores diagonal and tries to minimize off diagonal residualsCommunalites are derived from the solutionOriginally called Minimum Residual method (Comrey)Extraction MethodsGeneralized (weighted) least squares Also minimizes the off diagonal residualsVariables with larger communalities are given more weight in the analysisMany Other methodsRotation MethodsAfter extraction (regardless of method) good luck interpreting resultRotation is used to improve interpretability and utilityA orthogonally rotated solution is mathematically equivalent to un-rotated and other orthogonal solutionsStable and large N -> same resultGeometricRotationGeometric RotationFactor extraction equivalent to coordinate planesFactors are the axesLength of the line from the origin to the variable coordinates is equal to the communality for that variableOrthogonal Factors are at right anglesGeometric RotationFactor loadings are found by dropping a line from the variable coordinates to the factor at a right angleRepositioning the axes changes the loadings on the factor but keeps the relative positioning of the points the sameRotation MethodsOrthogonal vs. ObliqueOrthogonal rotation keeps factors un-correlated while increasing the meaning of the factorsOblique rotation allows the factors to correlate leading to a conceptually clearer picture but a nightmare for explanationRotation MethodsOrthogonal Rotation MethodsVarimax – most popular•Simple structure by maximizing variance of loadings within factors across variables•Makes large loading larger and small loadings smaller•Spreads the variance from first (largest) factor to other smaller factorsRotation MethodsOrthogonal Rotation MethodsQuartimax•Opposite of Varimax•Simplifies variables by maximizing variance with variables across factors•Varimax works on the columns of the loading matrix; Quartimax works on the rows•Not used as often; simplifying variables is not usually a goalRotation MethodsOrthogonal Rotation MethodsEquamax is a hybrid of the earlier two that tries to simultaneously simplify factors and variables•Not that popular eitherRotation MethodsOblique Rotation TechniquesDirect Oblimin•Begins with an unrotated solution •Has a parameter (gamma in SPSS) that allows the user to define