The Singular Value Decomposition Lecture 5 Introduction Amos Ron University of Wisconsin Madison February 03 2021 Amos Ron CS513 remote learning S21 The Singular Value Decomposition Outline 1 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Outline 1 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Blank page Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation SVD A is any matrix 2 2 De nition The SVD of A is a decomposition A U V cid 48 where U and V are orthogonal 2 2 is diagonal with non negative diagonal entries The columns of U are left singular vectors of A The columns of V are right singular vectors of A The diagonal entries of are singular values of A Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation SVD De nition The SVD of A is a decomposition A U V cid 48 where U and V are orthogonal 2 2 is diagonal with non negative diagonal entries The columns of U are left singular vectors of A The columns of V are right singular vectors of A The diagonal entries of are singular values of A Comment Let R be a diagonal matrix with unit diagonal entries Then if A U V cid 48 then also A UR RV cid 48 Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation SVD Comment Let R be a diagonal matrix with unit diagonal entries Then if A U V cid 48 then also A UR RV cid 48 Since UR and RV cid 48 are still orthogonal this is another SVD of A Up to this triviality i e the mulitiplication of the singular vectors by 1 the SVD is unique whenever 1 1 2 2 Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Reverse engineering on the SVD Assume that A U V cid 48 as before First fundamental computation AA cid 48 U V cid 48 U V cid 48 cid 48 U V cid 48 V U cid 48 U V cid 48 V U cid 48 U 2U cid 48 and A cid 48 A V 2V cid 48 Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Reverse engineering on the SVD Assume that A U V cid 48 as before First fundamental computation AA cid 48 U V cid 48 U V cid 48 cid 48 U V cid 48 V U cid 48 U V cid 48 V U cid 48 U 2U cid 48 and Conclusion A cid 48 A V 2V cid 48 The left singular vectors are the eigenvectors of AA cid 48 The right singular vectors are the eigenvectors of A cid 48 A The squares of the singular values are the eigenvalues of A cid 48 A and of AA cid 48 Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Reverse engineering on the SVD Assume that A U V cid 48 as before Second fundamental computation A cid 48 U V cid 48 cid 48 V U cid 48 So The left singular vectors of A are the right singular vectors of A cid 48 The left singular vectors of A cid 48 are the right singular vectors of A The singular values of A and A cid 48 are the same Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Reverse engineering on the SVD Assume that A U V cid 48 as before Third fundamental computation AV U V cid 48 V U i e Av1 1 1 u1 Av2 2 2 u2 Similarly A cid 48 U V Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation What are we really looking for The SVD reads as AV U It therefore nd two vector v1 v2 The columns of V such that v1 v2 is an orthonormal basis for IR2 Av1 Av2 0 Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation What are we really looking for The SVD reads as AV U It therefore nd two vector v1 v2 The columns of V such that v1 v2 is an orthonormal basis for IR2 Av1 Av2 0 Indeed if you nd such v1 v2 you may de ne ui Avi Avi 2 Then u1 u1 is an orthonormal basis too U is then their concatenation Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation What are we really looking for The SVD reads as AV U It therefore nd two vector v1 v2 The columns of V such that v1 v2 is an orthonormal basis for IR2 Av1 Av2 0 Indeed if you nd such v1 v2 you may de ne ui Avi Avi 2 AV U Then u1 u1 is an orthonormal basis too U is then their concatenation We get then with V U as above orthonormal then and diagonal with diagonal entries Avi 2 Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Derviation We take v1 v2 the orthonormal eigenbasis of A cid 48 A We then prove that Av1 Av2 are eigenvectors of AA cid 48 If their eigenvalues are different they must be perpendicular since AA cid 48 is symmetric Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Derviation We take v1 v2 the orthonormal eigenbasis of A cid 48 A We then prove that Av1 Av2 are eigenvectors of AA cid 48 If their eigenvalues are different they must be perpendicular since AA cid 48 is symmetric So i vi is an eigenpair of A cid 48 A and we want Avi to be an eigenpair of AA cid 48 AA cid 48 Avi A A cid 48 A vi A ivi iAvi Amos Ron CS513 remote learning S21 The Singular Value Decomposition De ning the SVD 2 2 case Derivation Derviation We take v1 v2 the orthonormal eigenbasis of A cid 48 A We then prove that Av1 Av2 are eigenvectors of AA cid 48 If their eigenvalues are different they must be perpendicular since AA cid 48 is symmetric So i vi is an eigenpair of A cid 48 A and we want Avi to be an eigenpair of AA cid 48 AA cid 48 Avi A A cid 48 A vi A ivi iAvi The above works if A is non sigular and 1 cid 54 …