Linear regression models in matrix termsThe regression function in matrix termsSimple linear regression functionSimple linear regression function in matrix notationDefinition of a matrixDefinition of a vector and a scalarMatrix multiplicationSlide 8Slide 9The Xβ multiplication in simple linear regression settingMatrix additionSlide 12The Xβ+ε addition in the simple linear regression settingMultiple linear regression function in matrix notationLeast squares estimates of the parametersLeast squares estimatesDefinition of the transpose of a matrixThe X'X matrix in the simple linear regression settingDefinition of the identity matrixDefinition of the inverse of a matrixLeast squares estimates in simple linear regression settingSlide 22Slide 23Least squares estimates in simple linear regression settingLinear dependenceLinear dependence is not always obviousImplications of linear dependence on regressionThe main point about linear dependenceImplications of linear dependence on regressionFitted values and residualsFitted valuesSlide 32The residual vectorThe residual vector written as a function of the hat matrixSum of squares and the analysis of variance tableAnalysis of variance table in matrix termsSum of squaresError sum of squaresSlide 39Total sum of squaresAn example of total sum of squaresSlide 42Model assumptionsError term assumptionsError terms as a random vectorThe mean (expectation) of the random error term vectorThe variance of the random error term vectorThe ASSUMED variance of the random error term vectorScalar by matrix multiplicationSlide 50The general linear regression modelLinear regression modelsin matrix termsThe regression function in matrix termsSimple linear regression functioniiixY10for i = 1,…, nnnnxYxYxY102210211101Simple linear regression function in matrix notationXYnnnxxxYYY21102121111Definition of a matrixAn r×c matrix is a rectangular array of symbols or numbers arranged in r rows and c columns. A matrix is almost always denoted by a single capital letter in boldface type.3621A9.14018.27115.26511.39214.3801B626152514241323122211211111111xxxxxxxxxxxxXDefinition of a vector and a scalarA column vector is an r×1 matrix, that is, a matrix with only one column.852qA row vector is an 1×c matrix, that is, a matrix with only one row. 90324621hA 1×1 “matrix” is called a scalar, but it’s just an ordinary number, such as 29 or σ2.Matrix multiplicationXY•The Xβ in the regression function is an example of matrix multiplication.•Two matrices can be multiplied together only if:–the # of columns of the first matrix equals the # of rows of the second matrix.•Then:–# of rows of the resulting matrix equals # of rows of first matrix.–# of columns of the resulting matrix equals # of columns of second matrix.Matrix multiplication•If A is a 2×3 matrix and B is a 3×5 matrix then matrix multiplication AB is possible. The resulting matrix C = AB has … rows and … columns.•Is the matrix multiplication BA possible?•If X is an n×p matrix and β is a p×1 column vector, then Xβ is …Matrix multiplication592738418810610190869637455123218791ABCThe entry in the ith row and jth column of C is the inner product (element-by-element products added together) of the ith row of A with the jth column of B.101)9(7)4(9)2(190)6(7)5(9)3(11211cc242327)6(2)7(1)1(8ccThe Xβ multiplication in simple linear regression setting1021111nxxxXMatrix additionXY•The Xβ+ε in the regression function is an example of matrix addition.•Simply add the corresponding elements of the two matrices.–For example, add the entry in the first row, first column of the first matrix with the entry in the first row, first column of the second matrix, and so on.•Two matrices can be added together only if they have the same number of rows and columns.Matrix addition14658510199812139257653781142BAC231211954972cccFor example:The Xβ+ε addition in the simple linear regression settingnnnxxxXYYYY211021011021Multiple linear regression functionin matrix notationXYnnnnnxxxxxxxxxYYY2132123222113121121111Least squares estimates of the parametersLeast squares estimates YXXXbbbbp1110The p×1 vector containing the estimates of the p parameters can be shown to equal:where (X'X)-1 is the inverse of the X'X matrix and X' is the transpose of the X matrix.Definition of the transpose of a matrixThe transpose of a matrix A is a matrix, denoted A' or AT, whose rows are the columns of A and whose columns are the rows of A … all in the same original order.978451ATAAThe X'X matrix in the simple linear regression settingnnxxxxxxXX1111112121Definition of the identity matrixThe (square) n×n identity matrix, denoted In, is a matrix with 1’s on the diagonal and 0’s elsewhere.10012IThe identity matrix plays the same role as the number 1 in ordinary arithmetic. 10016479Definition of the inverse of a matrixThe inverse A-1 of a square (!!) matrix A is the unique matrix such that …11 AAIAALeast squares estimates
View Full Document