The Art of Linear Algebra Graphic Notes on Linear Algebra for Everyone Kenji Hiranabe with the kindest help of Gilbert Strang September 1 2021 updated March 23 2023 Abstract I tried intuitive visualizations of important concepts introduced in Linear Algebra for Everyone 1 This is aimed at promoting understanding of vector matrix calculations and algorithms from the perspectives of matrix factorizations They include Column Row CR Gaussian Elimination LU Gram Schmidt Orthogonalization QR Eigenvalues and Diagonalization Q QT and Singular Value Decomposition U V T All the artworks including this article are maintained in the GitHub repository https github com kenjihiranabe The Art of Linear Algebra Foreword I am happy to see Kenji Hiranabe s pictures of matrix operations in linear algebra The pictures are an excellent way to show the algebra We can think of matrix multiplications by row column dot products but that is not all it is linear combinations and rank 1 matrices that complete the algebra and the art I am very grateful to see the books in Japanese translation and the ideas in Kenji s pictures Gilbert Strang Professor of Mathematics at MIT Contents 1 Viewing a Matrix 4 Ways 2 Vector times Vector 2 Ways 3 Matrix times Vector 2 Ways 4 Matrix times Matrix 4 Ways 5 Practical Patterns 6 The Five Factorizations of a Matrix 6 1 A CR 6 2 A LU 6 3 A QR 6 4 S Q QT 6 5 A U V T twitter hiranabe k hiranabe esm co jp https anagileway com Massachusetts Institute of Technology http www math mit edu gs 1 Linear Algebra for Everyone http math mit edu everyone with Japanese translation started by Kindai Kagaku 1 2 2 2 4 4 7 7 8 8 9 10 1 Viewing a Matrix 4 Ways A matrix m n can be seen as 1 matrix mn numbers n columns and m rows Figure 1 Viewing a Matrix in 4 Ways a11 a12 a21 a22 a31 a32 a1 a2 a a a 1 2 3 A Here the column vectors are in bold as a1 Row vectors include as in a 1 Transposed vectors and matrices are indicated by T as in aT and AT 2 Vector times Vector 2 Ways Hereafter I point to speci c sections of Linear Algebra for Everyone and present graphics which illustrate the concepts with short names in colored circles Sec 1 1 p 2 Linear combination and dot products Sec 1 3 p 25 Matrix of Rank One Sec 1 4 p 29 Row way and column way Figure 2 Vector times Vector v1 v2 v1 is a elementary operation of two vectors but v2 multiplies the column to the row and produce a rank 1 matrix Knowing this outer product v2 is the key for the later sections 3 Matrix times Vector 2 Ways A matrix times a vector creates a vector of three dot products Mv1 as well as a linear combination Mv2 of the column vectors of A 2 01 2 1 0 2 3 2 4 5 6 01 2 3 41 56 57 1 1 56 8 9 5 1 7 57 1 1 2 3 1 568 57 6 77 17 5 1 56 1 1 1 5 7 1 8 Sec 1 1 p 3 Linear combinations Sec 1 3 p 21 Matrices and Column Spaces Figure 3 Matrix times Vector Mv1 Mv2 At rst you learn Mv1 But when you get used to viewing it as Mv2 you can understand Ax as a linear combination of the columns of A Those products ll the column space of A denoted as C A The solution space of Ax 0 is the nullspace of A denoted as N A Also vM1 and vM2 shows the same patterns for a row vector times a matrix Figure 4 Vector times Matrix vM1 vM2 The products ll the row space of A denoted as C AT The solution space of yA 0 is the left nullspace of A denoted as N AT The four subspaces consists of N A C AT which are perpendicular to each other in Rn and N AT C A in Rm which are perpendicular to each other Sec 3 5 p 124 Dimensions of the Four Subspaces 3 01 2312 4 56 7 84 5 4 93 40 1 8 7 3 40 72 128 528 2 8 10 8 0 1 2 0 34 20 2 5 6 7 5 898 6 7 3 1 1 4 6 1 32 7 5 802 4 3 6 1 3 2 5 9 Figure 5 The Four Subspaces See A CR Sec 6 1 for the rank r 4 Matrix times Matrix 4 Ways Matrix times Vector naturally extends to Matrix times Matrix Sec 1 4 p 35 Four Ways to Multiply AB C Also see the back cover of the book Figure 6 Matrix times Matrix MM1 MM2 MM3 MM4 5 Practical Patterns Here I show some practical patterns which allow you to capture the coming factorizations more intuitively 4 01 01 0 1 0 2 13 0 0 0 0 1 1 2 2 1 2 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3456 37869 48376 43 4 6 A6 9B 4 6 A6CD 1 1 1 E 1 E1 E1 EFB9 G9 3 46 H 6 7 I8465 J46 636 B 6 A6734I6 6 3 7 8 DKL86G7 5B 8567 J 637869 48376 43 4 6 A67 J DMN87O6898 84 6 8 8 6365 6G7 5B 6 A67 J6N8 73456 9B 46N8 7D Figure 7 Pattern 1 2 P1 P1 Pattern 1 is a combination of MM2 and Mv2 Pattern 2 is an extention of MM3 Note that Pattern 1 is a column operation multiplying a matrix from right whereas Pattern 2 is a row operation multiplying a matrix from left Figure 8 Pattern 1 2 P1 P2 P1 multipies the diagonal numbers to the columns of the matrix whereas P2 multipies the diagonal numbers to the row of the matrx Both are variants of P1 and P2 Figure 9 Pattern 3 P3 This pattern appears when you solve di erential equations and recurrence equations Sec 6 p 201 Eigenvalues and Eigenvectors Sec 6 4 p 243 Systems of Di erential Equations 5 0 0 12 34 5 3 0 6 1 0 6 21 4 1 0 7 34 5 3 0 6 1 0 6 21 4 0 12 1 34 23 2 41 4 5 6 0 12 20 34 23 2 41 76 0 1 23 45 3 6 22 3 7 11 28 3 93 4 In both cases the solutions are expressed with eigenvalues 1 2 3 eigenvectors X x1 x2 x3 which are the coordinates of the initial condition u 0 u0 in c1 c2 c3 of A and the coe cients c terms of the eigenvectors X du t Au t u 0 u0 dt un 1 Aun u0 u0 T u0 c1x1 c2x2 c3x3 X c1 c2 c3 c 1u0 and the general solution of the two equations …