1Chapter 5Dimension Reduction and Extraction of Meaningful Factors Section 5.1Principal Components Analysis3Objectives  Explain the basic concepts for principal components analysis. Identify several strategies for selecting the number of components. Perform principal components analysis using the PRINCOMP procedure.4Too Many VariablesSystolic bloodpressureDiastolicbloodpressureDietExerciseLDL CholesterolHDL CholesterolMedication5Solutions Eliminate some redundant variables.– May lose important information that was uniquely reflected in the eliminated variables. Create composite scores from variables (sum or average).– Lost variability among the variables– Multiple scale scores may still be collinear Create weighted linear combinations of variables while retaining most of the variability in the data.– Fewer variables; little or no lost variation– No collinear scales.6An Easy ChoiceTo retain most of the information in the data while reducing the number of variables you must deal with, try principal components analysis.  Most of the variability in the original data can be retained.but… Components may not be directly interpretable.27Principal Components AnalysisPCA is a dimension reduction method that creates variables called principal components creates as many components as there are input variables.8Principal ComponentsPrincipal components are weighted linear combinations of input variables are orthogonal to and independent of other components are generated so that the first component accounts for the most variation in the xs, followed by the second component, and so on.9First Principal Component10Second Principal Component11More on the Geometric PropertiesLeast squares regression minimizes the sum of squared vertical distances to the fitted line (perpendicular to x).y1y2..................PCA minimizes the sum of the squared perpendiculardistances to the axis of the PC. 12Details of Principal Components The j principal components provide a least-squares solution to the following model:Y = XBwhereY n by p matrix of scores on the componentsX n by j matrix of centered observed variablesB j by p matrix of eigenvectors of the correlation or covariance matrix of the variables.313How Many Components? Scree plot of eigenvalues: Proportion of varianceexplained by each component: Cumulative variance explained by components: Eigenvalue > 1******1212or .. ( )..()iipktrtrλλλλ λλλ λ++++++RR14Principal Component ScoresPrincipal component scores can be created  for each observation in the data set  on each principal component  using raw or the standardized weights.15Graphical Exploration of the PCs16Assumptions of PCA Random missingness Absence of outliers Singularity not a mathematical problem in PCA because matrices are not inverted.17Procedures That Can Perform PCA PRINCOMP PRINQUAL CORRESP PLS FACTOR.18The PRINCOMP ProcedureGeneral form of the PRINCOMP procedure:PROC PRINCOMP <options>;VAR variables;RUN;PROC PRINCOMP <options>;VAR variables;RUN;419The FACTOR ProcedureGeneral form of the FACTOR procedure:PROC FACTOR options;VAR variables;RUN;PROC FACTOR options;VAR