New version page

PSU STAT 501 - The regression function in matrix terms

Documents in this Course
VARIABLES

VARIABLES

33 pages

Load more

This preview shows page 1-2-3-24-25-26-27-49-50-51 out of 51 pages.

View Full Document
View Full Document

End of preview. Want to read all 51 pages?

Upload your study docs or become a GradeBuddy member to access this document.

View Full Document
Unformatted text preview:

Linear regression models in matrix termsThe regression function in matrix termsSimple linear regression functionSimple linear regression function in matrix notationDefinition of a matrixDefinition of a vector and a scalarMatrix multiplicationSlide 8Slide 9The Xβ multiplication in simple linear regression settingMatrix additionSlide 12The Xβ+ε addition in the simple linear regression settingMultiple linear regression function in matrix notationLeast squares estimates of the parametersLeast squares estimatesDefinition of the transpose of a matrixThe X'X matrix in the simple linear regression settingDefinition of the identity matrixDefinition of the inverse of a matrixLeast squares estimates in simple linear regression settingSlide 22Slide 23Least squares estimates in simple linear regression settingLinear dependenceLinear dependence is not always obviousImplications of linear dependence on regressionThe main point about linear dependenceImplications of linear dependence on regressionFitted values and residualsFitted valuesSlide 32The residual vectorThe residual vector written as a function of the hat matrixSum of squares and the analysis of variance tableAnalysis of variance table in matrix termsSum of squaresError sum of squaresSlide 39Total sum of squaresAn example of total sum of squaresSlide 42Model assumptionsError term assumptionsError terms as a random vectorThe mean (expectation) of the random error term vectorThe variance of the random error term vectorThe ASSUMED variance of the random error term vectorScalar by matrix multiplicationSlide 50The general linear regression modelLinear regression modelsin matrix termsThe regression function in matrix termsSimple linear regression functioniiixY10for i = 1,…, nnnnxYxYxY102210211101Simple linear regression function in matrix notationXYnnnxxxYYY21102121111Definition of a matrixAn r×c matrix is a rectangular array of symbols or numbers arranged in r rows and c columns. A matrix is almost always denoted by a single capital letter in boldface type.3621A9.14018.27115.26511.39214.3801B626152514241323122211211111111xxxxxxxxxxxxXDefinition of a vector and a scalarA column vector is an r×1 matrix, that is, a matrix with only one column.852qA row vector is an 1×c matrix, that is, a matrix with only one row. 90324621hA 1×1 “matrix” is called a scalar, but it’s just an ordinary number, such as 29 or σ2.Matrix multiplicationXY•The Xβ in the regression function is an example of matrix multiplication.•Two matrices can be multiplied together only if:–the # of columns of the first matrix equals the # of rows of the second matrix.•Then:–# of rows of the resulting matrix equals # of rows of first matrix.–# of columns of the resulting matrix equals # of columns of second matrix.Matrix multiplication•If A is a 2×3 matrix and B is a 3×5 matrix then matrix multiplication AB is possible. The resulting matrix C = AB has … rows and … columns.•Is the matrix multiplication BA possible?•If X is an n×p matrix and β is a p×1 column vector, then Xβ is …Matrix multiplication592738418810610190869637455123218791ABCThe entry in the ith row and jth column of C is the inner product (element-by-element products added together) of the ith row of A with the jth column of B.101)9(7)4(9)2(190)6(7)5(9)3(11211cc242327)6(2)7(1)1(8ccThe Xβ multiplication in simple linear regression setting1021111nxxxXMatrix additionXY•The Xβ+ε in the regression function is an example of matrix addition.•Simply add the corresponding elements of the two matrices.–For example, add the entry in the first row, first column of the first matrix with the entry in the first row, first column of the second matrix, and so on.•Two matrices can be added together only if they have the same number of rows and columns.Matrix addition14658510199812139257653781142BAC231211954972cccFor example:The Xβ+ε addition in the simple linear regression settingnnnxxxXYYYY211021011021Multiple linear regression functionin matrix notationXYnnnnnxxxxxxxxxYYY2132123222113121121111Least squares estimates of the parametersLeast squares estimates YXXXbbbbp1110The p×1 vector containing the estimates of the p parameters can be shown to equal:where (X'X)-1 is the inverse of the X'X matrix and X' is the transpose of the X matrix.Definition of the transpose of a matrixThe transpose of a matrix A is a matrix, denoted A' or AT, whose rows are the columns of A and whose columns are the rows of A … all in the same original order.978451ATAAThe X'X matrix in the simple linear regression settingnnxxxxxxXX1111112121Definition of the identity matrixThe (square) n×n identity matrix, denoted In, is a matrix with 1’s on the diagonal and 0’s elsewhere.10012IThe identity matrix plays the same role as the number 1 in ordinary arithmetic. 10016479Definition of the inverse of a matrixThe inverse A-1 of a square (!!) matrix A is the unique matrix such that …11  AAIAALeast squares estimates


View Full Document
Loading Unlocking...
Login

Join to view The regression function in matrix terms and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view The regression function in matrix terms and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?