ZOOMNOTES FOR LINEAR ALGEBRA GILBERT STRANG Massachusetts Institute of Technology WELLESLEY CAMBRIDGE PRESS Box 812060 Wellesley MA 02482 ZoomNotes for Linear Algebra Copyright 2021 by Gilbert Strang ISBN 978 1 7331466 4 7 LATEX typesetting by Ashley C Fernandes Printed in the United States of America 9 8 7 6 5 4 3 2 1 Texts from Wellesley Cambridge Press Linear Algebra for Everyone 2020 Gilbert Strang Linear Algebra and Learning from Data 2019 Gilbert Strang Introduction to Linear Algebra 5th Ed 2016 Gilbert Strang Computational Science and Engineering Gilbert Strang Differential Equations and Linear Algebra Gilbert Strang Wavelets and Filter Banks Gilbert Strang and Truong Nguyen Introduction to Applied Mathematics Gilbert Strang Calculus Third Edition Gilbert Strang Algorithms for Global Positioning Kai Borre Gilbert Strang Essays in Linear Algebra Gilbert Strang An Analysis of the Finite Element Method 2008 edition Gilbert Strang and George Fix ISBN 978 1 7331466 3 0 ISBN 978 0 6921963 8 0 ISBN 978 0 9802327 7 6 ISBN 978 0 9614088 1 7 ISBN 978 0 9802327 9 0 ISBN 978 0 9614088 7 9 ISBN 978 0 9614088 0 0 ISBN 978 0 9802327 5 2 ISBN 978 0 9802327 3 8 ISBN 978 0 9802327 6 9 ISBN 978 0 9802327 0 7 Wellesley Cambridge Press Box 812060 Wellesley MA 02482 USA www wellesleycambridge com Gilbert Strang s page math mit edu gs For orders math mit edu weborder php Outside US Canada www cambridge org Select books India www wellesleypublishers com The textbook websites are math mit edu linearalgebra and math mit edu everyone Those sites link to 18 06 course materials and video lectures on YouTube and OCW Solution Manuals can be printed from those sites and math mit edu learningfromdata Linear Algebra is included in MIT s OpenCourseWare site ocw mit edu courses This provides video lectures of the full linear algebra courses 18 06 and 18 06 SC and 18 065 ZoomNotes for Linear Algebra Gilbert Strang Preface Textbooks ZoomNotes and Video Lectures Three Great Factorizations LU and QR and SVD Part 1 Basic Ideas of Linear Algebra Part 2 Solving Linear Equations Ax b A is n by n Part 3 Vector Spaces and Subspaces Basis and Dimension Part 4 Orthogonal Matrices and Least Squares Part 5 Determinant of a Square Matrix Part 6 Eigenvalues and Eigenvectors Ax x and Anx nx Part 7 Singular Values and Vectors Av u and A U V T Part 8 Linear Transformations and Their Matrices Part 9 Complex Numbers and the Fourier Matrix Part 10 Learning from Data Minimize Loss by Gradient Descent Part 11 Basic Statistics Mean Variance Covariance iii 1 2 3 5 14 21 30 35 40 46 54 59 65 72 Preface The title ZoomNotes indicates that these pages were created in 2020 and 2021 But they are not limited to online lectures I hope these notes will help instructors and students to see linear algebra in an organized way from vectors to matrices to subspaces to bases Linear independence is a crucial idea for this subject so it comes early for vectors of integers I hope that faculty who are planning a linear algebra course and students who are reading for themselves will see these notes A happy part of linear algebra is the wonderful variety of matrices diagonal triangular symmetric orthogonal and many more The organizing principles have become matrix factoriza tions like A LU lower triangular times upper triangular The idea of elimination to simplify the equations Ax b by introducing zeros in the matrix appears early as it must Please don t spend forever on those computations Linear algebra has so many more good ideas The reader may know my video lectures on OpenCourseWare Math 18 06 is on ocw mit edu and on Youtube mitocw I am so grateful that those have been helpful Now I have realized that lecture notes can help in a different way You will quickly gain a picture of the whole course the structure of the subject the key topics in a natural order the connecting ideas that make linear algebra so beautiful This structure is the basis of two textbooks from Wellesley Cambridge Press Introduction to Linear Algebra Linear Algebra for Everyone I don t try to teach every topic in those books I do try to reach eigenvalues and singular values A basis of eigenvectors for square matrices and of singular vectors for all matrices takes you to the heart of a matrix in a way that elimination cannot do The last chapters of these notes extend to a third book and a second math course 18 065 with videos on OpenCourseWare Linear Algebra and Learning from Data Wellesley Cambridge Press 2019 This is Deep Learning and it is not entirely linear It creates a learning function F x v from training data v like images of handwritten numbers and matrix weights x The piecewise linear ReLU function plays a mysterious but crucial part in F Then F x vnew can come close to new data that the system has never seen The learning function F x v grows out of linear algebra and optimization and statistics and high performance computing Our aim is to understand in part why it succeeds Above all I hope these ZoomNotes help you to teach linear algebra and learn linear algebra This subject is used in so many valuable ways And it rests on ideas that everyone can understand Thank you Gilbert Strang Textbooks ZoomNotes and Video Lectures Introduction to Linear Algebra 5th Ed 2016 Linear Algebra and Learning from Data 2019 Linear Algebra for Everyone 2020 Differential Equations and Linear Algebra 2014 ZoomNotes for Linear Algebra 2021 math mit edu linearalgebra math mit edu learningfromdata math mit edu everyone math mit edu dela Video Lectures OpenCourseWare ocw mit edu courses youtube mitocw Math 18 06 and 18 06SC Linear Algebra at MIT A 2020 Vision of Linear Algebra added to 18 06 Linear Algebra and Learning from Data Math 18 065 Math 18 085 and 18 086 Computational Science and Engineering Strang and Moler Interview with Lex Fridman https www youtube com watch v lEZPfmGCEk0 Differential Equations and Linear Algebra Wellesley Cambridge Press Box 812060 Wellesley MA 02482 USA www wellesleycambridge com Orders math mit edu weborder php Gilbert Strang s page math mit edu gs Outside US Canada www cambridge org Select books India www wellesleypublishers com Three Great Factorizations LU QR SVD Orthogonal matrix QTQ I Square QQT I Triangular matrix Rij 0 for i j Rjj 6 0 on diagonal Q q1 q2 R Orthogonal basis r11 r12 r22 r1n r2n rnn Triangular basis qn 1 A LU lower triangular upper triangular Elimination 2 A QR orthogonal upper triangular Gram Schmidt 3 A U V T orthogonal diagonal orthogonal