1Chapter 5Dimension Reduction and Extraction of Meaningful Factors 2The FACTOR ProcedureGeneral form of the FACTOR procedure:PROC FACTOR options;VAR variables;RUN;PROC FACTOR options;VAR variables;RUN;Section 5.2Exploratory Factor Analysis4Objectives Explain the distinctions between principal components and common factor analyses. Identify several factor extraction methods for factor analysis. Differentiate between oblique and orthogonal rotation methods for factor analysis. Use the FACTOR procedure to perform exploratory factor analysis.5Why Perform Factor Analysis?You suspect that the variables you observe (manifest variables) are functions of variables that you cannot observe directly (latent variables). Identify the latent variables to learn something interesting about the behavior of your population. Identify relationships between different latent variables. Show that a small number of latent variables underlies the process or behavior you have measured to simplify your theory. Explain inter-correlations among observed variables.6Exploratory Factor AnalysisF1:ConsumerconfidenceF2: BuyingpowerNew Home BuysDurable Goods BuysBorrowingIncomeImport Purchasesu1u2u3u4u5?27Components versus Factors, RevisitedPrincipal Components –the symptomsLatent Factors –the disease8The Common Factor ModelY = X β + Ewhere Y manifest variables X common factors β weights E unique factors + error variation9Assumptions of the Common Factor Model The unique factors (residuals) are uncorrelated with each other. The unique factors (residuals) are uncorrelated with the common (latent) factors.Under these constraints, you can solve for the correlation matrix: or ′′R=ββ+U R-U=ββ10PCA versus Factor AnalysisThe variables reflect the common (latent) factors and explain shared variation in the manifest variables.The components are derived from the variables and explain 100% of the variation in the data.Not necessary that 100% of variance be accounted for by the extracted factors.100% of variance accounted for by all components.Factor AnalysisPCA11Limitations of Exploratory Factor AnalysisFactor scores are not linear combinations of the variables. They are estimates of latent factors. Try to avoid data fishing problems by:1. Carefully selecting your manifest variables.2. Using rotation to interpret the factors.3. Performing a confirmatory factor analysis to test hypotheses about the adequacy of the factor solution.12Factor Extraction Methods: OverviewPrincipal Factor Analysis Computationally efficient Most commonly used.Maximum Likelihood Factor Analysis Less efficient computationally; iterative procedure Better estimates than principal factor analysis in large samples Hypothesis tests for number of factors.Prior communality estimates are usually the squared multiple correlation of each manifest variable with all the others.313How Many Factors? Proportion of variance accounted for– Minimum # factors to account for 100% of the common variance Scree test– Find the elbow Interpretability criteria– At least three items load on each factor– Variables within a factor share conceptual meaning– Variables between factors measure different constructs– Rotated factors demonstrate simple structure.14The FACTOR Procedure, revisitedGeneral form of the FACTOR procedure:PROC FACTOR options;VAR variables;RUN;PROC FACTOR options;VAR variables;RUN;15This demonstration illustrates exploratory factor analysis using the FACTOR procedureUsing the FACTOR Procedurech5s2d1.sas16Are Factors Correlated? Rotation MethodsBuyingPowerConsumerConfidenceBuyingPowerConsumerConfidenceOrthogonalOblique17Rotation Methods Varimax-Orthogonal: Maximizes the variance of columns of the factor pattern matrix.  Promax-Oblique in two steps: 1. Varimax rotation 2. Relax orthogonality constraints and rotate further.PROC FACTOR can also perform many other rotation methods. See the SAS/STAT User’s Guide. 18Displayed Factor Analysis OutputEigenvalues In factor analysis, the eigenvalues displayed are related to the reduced correlation matrix (R-U). In PCA, eigenvalues are of R.  Rule of eigenvalue > 1 is less meaningful in determining the number of factors to retain for factor analysis. Scree plot of eigenvalues is often useful in factor analysis.419Displayed Factor Analysis OutputFactor Pattern Matrix The matrix of standardized regression coefficients for Y = XB + E Equal to the matrix of correlations between the variables and the extracted (orthogonal) common factors.20Displayed Factor Analysis OutputRotated Factor Pattern Matrix The matrix of standardized regression coefficients for rotated factors Equal to the correlations between the variables and the rotated common factors for orthogonal rotations.21Displayed Factor Analysis OutputStructure Matrix Generated for oblique rotations only  The matrix of the correlations between variables and rotated common factors.22Displayed Factor Analysis OutputReference Structure Matrix Generated for oblique rotations only  The matrix of semipartial correlations between variables and common factors, removing from each common factor the effects of other common factors.23Displayed Factor Analysis OutputCorrelation between Factors Generated for oblique rotations only Factor plotsFinal communality estimates R2for predicting variables from factors Called squared canonical correlations when ML method is usedVariance explained by each factor24This demonstration illustrates the FACTOR procedure for exploratory factor analysis with rotation.Rotated Factor Analysis ch5s2d2.sas525This exercise reinforces the concepts discussed previously.ExercisesSection 5.3Cronbach’s Coefficient Alpha for Scale Reliability27Objectives Perform reliability analysis of scale data using Cronbach’s coefficient alpha. Interpret output from the CORR procedure for Cronbach’s alpha.28Internal Consistency of a ScaleWhen measuring a latent variable, you need a way to quantify the latent variable.Items that load on a factor for the latent variable are often summed to create a scale score. But how reliable is the scale?The “true” reliability is the squared correlation between the scale score (Y) and the true value of the latent variable (T).29Cronbach’s AlphaOne way of estimating the reliability of a scale is to compute Cronbach’s alphawhere Y are the variables that make up the scale, p is the