EXST7015 Matrix Algebra Geaghan Matrix (Part 1) Introduction & Simple Linear Regression Page 1 06d-MatrixAlgebraIntro.doc A. MATRIX STRUCTURE AND NOTATION 1) A matrix is a rectangular arrangement of numbers. The matrix is usually denoted by a capital letter. A = 1379LNMOQP D = 424160305230LNMMMMOQPPPP 2) The dimensions of a matrix are given by the number of rows and columns in the matrix (i.e. the dimensions are r by c). For the matrices above, A is 2 by 2 D is 4 by 3 3) The individual elements of a matrix can be referred to by specifying the row and column in which it occurs. Lower case numbers are used to represent individual elements, and should match the upper case letter used to denote matrix. For example, individual elements from matrices A and D above can be referred to as, a11 = 1 a21 = 7 d22 = 6 d12 = 2 B. TYPES OF MATRICES 1) Square matrix - the number of rows and columns are equal. Matrix A above is a square matrix (2 by 2), matrix D is not (4 by 3). A symmetric matrix is an important variation of the square matrix. In a symmetric matrix, the value in position “ij" equals the value in position “ji" (where i ≠ j). For example, if c31 = 5 then c13 is also 5. 2) Scalar - a single number can be thought of as a 1 by 1 matrix and is called a scalar. 3) Vector - a single column or single row of numbers is called a vector. The dimensions of a row vector are (1 by c), where "c" is the number of columns, and the dimensions of a column vector (r by 1), where "r" is the number of rows. 4) Identity matrix - this special square matrix consists of all ones on the main diagonal, or principal diagonal, and zeros in all the off diagonal positions. The following are examples of identity matrices, E = 100010001LNMMMOQPPP F = 1000010000100001LNMMMMOQPPPP The diagonal matrix is a generalization of the identity matrix. A diagonal matrix can have any value on the main diagonal, but also has zeros in the off diagonal positions.EXST7015 Matrix Algebra Geaghan Matrix (Part 1) Introduction & Simple Linear Regression Page 2 06d-MatrixAlgebraIntro.doc C. MATRIX TRANSPOSE The transpose of a matrix consists of a new matrix such that the rows of the original matrix become the columns of the transpose matrix. The transpose matrix is denoted with the same letter as the original matrix followed by a prime (e.g. the transpose of X is X). D = 424160305230LNMMMMOQPPPP D = 413226034050LNMMMOQPPP D. MATRIX ADDITION AND SUBTRACTION Matrices to be added or subtracted must be of the same dimensions. Each element of the first matrix, (a) is added (or subtracted) from the corresponding element of the second matrix, (b). A = 123490−LNMMMOQPPP B = 141444−LNMMMOQPPP A+B = 11 2 431 4 494 04+−+++−+LNMMMOQPPP = 224854LNMMMOQPPP E. MATRIX MULTIPLICATION Multiplication by a scalar - in this type of multiplication each element of the matrix is simply multiplied, element by element, by the scalar value. A = 123490−LNMMMOQPPP B = [7] A * B = 7 * 123490−LNMMMOQPPP= 71421 2863 0−LNMMMOQPPP Element by element multiplication - matrix multiplication is not usually done by matching each i,jth element of one matrix with the corresponding ijth element of the second matrix. This is called elementwise multiplication and it is not the normal mode of matrix multiplication and should not be used unless specifically requested. The standard method of matrix multiplication requires that the number of columns in the first matrix equal the number of rows in the second matrix. If the first matrix is (r by c) and the second is (r by c), in order to multiply the matrices, c must equal r. The resulting matrix will have the dimensions (r by c). Multiplication is accomplished by summing the cross products of each row of the first matrix and each column of the second matrix. A = 123490−LNMMMOQPPP X = 1234−LNMOQP Since A is 3 rows by 2 columns, and X is 2 by 2, then the columns of the first matrix equals the rows of the second matrix, and the matrices may be multiplied. A*X = 123490−LNMMMOQPPP * 1234−LNMOQP = (1*1) + (-2 * 3) (1* -2) + (-2 * 4)(3*1) + (4 * 3) (3* -2) + (4 * 4)(9*1)+(0*3) (9*-2)+(0*4)LNMMMOQPPP = −−−LNMMMOQPPP51015 10918EXST7015 Matrix Algebra Geaghan Matrix (Part 1) Introduction & Simple Linear Regression Page 3 06d-MatrixAlgebraIntro.doc the new dimensions for the product of A * X are, ↓ must be equal ↓ (3 x 2) x (2 x 3) ↑ new dimensions ↑ Note that though we can multiply A * X, we could not have done the multiplication the other way (i.e. X * A), since the dimensions would not have matched. That is, we could pre-multiply by A, but could not pre-multiply by X. F. SIMPLE MATRIX INVERSION (2 by 2 matrix only) Matrices are not “divided", but may be inverted. Instead of “dividing" A by B, one would multiply A by the inverse of B. The inverse of a (2 by 2) matrix is given by, A =abcd⎡⎤⎢⎥⎣⎦ A–1 = ()()d-b1-c aa×b - b×c⎡⎤⎢⎥⎣⎦ The scalar value resulting from the calculation “(a%d) – (b%c)" is called the determinant. The matrix cannot be inverted unless the inverse of the determinant exists (is defined). It will not exist in a case such as the one below since (1+0) is not defined. A =1428LNMOQP A–1 = 1118 2 410Determinant of A=×−×=afaf This occurs in regression when two variables are linearly related. An example of the inversion of a 2 * 2 matrix is given below. B =2314LNMOQP B–1 = 124 13431215431208 0602 0×−×−−LNMOQP=−−LNMOQP=−−LNMOQPafaf....4 Note that a matrix times its inverse (i.e. B % B–1) results in an identity matrix. By definition, the inverse of a matrix G is a matrix which when multiplied by G produces an identity matrix, or G%G–1=I. G. SIMPLE LINEAR REGRESSION Solving a simple linear regression with matrices requires the same values used for an algebraic solution from summation notation formulas. These are; n , nii=1X∑ , nii=1Y∑ , n2ii=1X∑ , n2ii=1Y∑ , niii=1XY∑ where n is the size of the sample of data. To obtain these values in the matrix form we start with the matrix equivalent of the individual values of X and Y, the raw data matrices.EXST7015 Matrix Algebra Geaghan Matrix (Part 1) Introduction & Simple Linear Regression Page 4 06d-MatrixAlgebraIntro.doc XXXXXXXX1234567=LNMMMMMMMMMOQPPPPPPPPP1111111 YYYYYYYY1234567=LNMMMMMMMMMOQPPPPPPPPP The column of