Chapter 5: Matrix approach to simple linear regression analysis You need to understand matrix algebra for multiple regression! Fox’s Section 8.2 contains information abouthow to use R for matrix algebra. 5.1 Matrices What is a matrix? “A matrix is a rectangular array of elements arranged in rows and columns” (p. 176 of KNN)Example: 1 2 34 5 6   Dimension – Size of matrix: # rows  # columns = rcExample: 23Symbolic representation of a matrix: Example: 11 12 1321 22 23a a aa a a   A  2012 Christopher R. Bilder5.1where aij is the row i and column j element of Aa11=1 from the above exampleNotice that the matrix A is in bold. When bolding is not possible (writing on a piece of paper or chalkboard), the letter is underlined - A a11 is often called the “(1,1) element” of A, a12 is called the “(1,2) element” of A,…Example: rc matrix11 12 1j 1c21 22 2 j 2ci1 i2 ij icr1 r 2 rj rca a a aa a a aa a a aa a a a          AL LL LM M M ML LM M M ML LExample: Square matrix is rc where r=cExample: HS and College GPA 2012 Christopher R. Bilder5.21 3.041 2.351 2.701 2.281 1.88          XM MThe above 202 matrix contains the HS GPAs in thesecond column. Vector – a r1 (column vector) or 1c (row vector) matrix – special case of a matrixExample: Symbolic representation of a 31 column vector123aaa     AExample: HS and College GPA3.102.303.002.201.60          YMThe above 201 vector contains the College GPAs. 2012 Christopher R. Bilder5.3Transpose: Interchange the rows and columns of a matrix or vectorExample:11 12 1321 22 23a a aa a a   A and 11 2112 2213 23a aa aa a� �� ��=� �� �� �AA is 23 and A is 32The  symbol indicates a transpose, and it is said as the word “prime”. Thus, the transpose of A is “A prime”.Example: HS and College GPA 3.10 2.30 3.00 2.20 1.60Y L 2012 Christopher R. Bilder5.45.2 Matrix addition and subtractionAdd or subtract the corresponding elements of matrices with the same dimension. Example:Suppose 1 2 3 1 10 1 and 4 5 6 5 5 8           A B. Then0 12 2 2 8 4 and 9 10 14 1 0 2             A B A B.Example: Using R (basic_matrix_algebra.R)> A<-matrix(data = c(1, 2, 3, 4, 5, 6), nrow = 2, ncol = 3, byrow = TRUE)> class(A)[1] "matrix"> B<-matrix(data = c(-1, 10, -1, 5, 5, 8), nrow = 2, ncol = 3, byrow = TRUE) > A+B [,1] [,2] [,3][1,] 0 12 2[2,] 9 10 14> A-B [,1] [,2] [,3][1,] 2 -8 4[2,] -1 0 -2 2012 Christopher R. Bilder5.5Notes: 1.Be careful with the byrow option. By default, this is setto FALSE. Thus, the numbers would be entered into the matrix by columns. For example,> matrix(data = c(1, 2, 3, 4, 5, 6), nrow = 2, ncol = 3) [,1] [,2] [,3][1,] 1 3 5[2,] 2 4 62.The class of these objects is “matrix”. 3.A vector can be represented as a “matrix” class type ora type of its own. > y<-matrix(data = c(1,2,3), nrow = 3, ncol = 1, byrow = TRUE)> y [,1][1,] 1[2,] 2[3,] 3> class(y)[1] "matrix"> x<-c(1,2,3)> x[1] 1 2 3> class(x)[1] "numeric"> is.vector(x)[1] TRUEThis can present some confusion when vectors are multiplied with other vectors or matrices because no specific row or column dimensions are given. More onthis shortly.  2012 Christopher R. Bilder5.64.A transpose of a matrix can be done using the t() function. For example, > t(A) [,1] [,2][1,] 1 4[2,] 2 5[3,] 3 6 Example: Simple linear regression modelYi=E(Yi) + i for i=1,…,n can be represented as E( ) Y Y  where 1 1 12 2 2n n nY E(Y )Y E(Y ), E( ) , and Y E(Y )e� � � � � �� � � � � �e� � � � � �= = =� � � � � �� � � � � �e� � � � � �M M MY Y e 2012 Christopher R. Bilder5.75.3 Matrix multiplicationScalar - 11 matrixExample: Matrix multiplied by a scalar11 12 1321 22 23ca ca cacca ca ca   A where c is a scalarLet 1 2 34 5 6   A and c=2. Then 2 4 628 10 12   A.Multiplying two matricesSuppose you want to multiply the matrices A and B; i.e., AB or AB. In order to do this, you need the number of columns of A to be the same as the number of rows as B. For example, suppose A is 23 and B is 310. You can multiply these matrices. However if B is 410 instead, these matrices could NOT be multiplied. The resulting dimension of C=AB1.The number of rows of A is the number of rows of C.2.The number of columns of B is the number of rows of C.3.In other words, w y w z z y  C A B where the dimension of the matrices are shown below them.  2012 Christopher R. Bilder5.8How to multiply two matrices – an exampleSuppose 3 01 2 3 and 1 24 5 60 1         A B. Notice that A is 23 and B is 32 so C=AB can be done. 3 01 2 31 24 5 60 11 3 2 1 3 0 1 0 2 2 3 14 3 5 1 6 0 4 0 5 2 6 15 717 16                                 C ABThe “cross product” of the rows of A and the columns of B are taken to form CIn the above example, D=BAAB where BA is: 2012 Christopher R. Bilder5.93 01 2 31 24 5 60 13 1 0 4 3 2 0 5 3 3 0 61 1 2 4 1 2 2 5 1 3 2 60 1 1 4 0 2 1 5 0 3 1 63 6 99 12 154 5 6� �� �� �=� �� �� �� �� �* + * * + * * + *� �� �= * + * * + * * + *� �� �* + * * + * * + *� �� �� �=� �� �� �BAIn general for a 23 matrix times a 32 matrix: 11 1211 12 1321 2221 22 2331 3211 11 12 21 13 31 11 12 12 22 13 3221 11 22 21 23 31 21 12 22 22 23 32b ba a ab ba a ab ba b a b a b a b a b a ba b a b a b a b a b a b          …