The Singular Value DecompositionDefining the SVD, 22 caseDerivationThe Singular Value DecompositionLecture 5: IntroductionAmos RonUniversity of Wisconsin - MadisonFebruary 03, 2021Amos Ron CS513, remote learning S21The Singular Value DecompositionOutline1The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationAmos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationOutline1The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationAmos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationBlank pageAmos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationSVDA is any matrix 2 × 2.DefinitionThe SVD of A is a decompositionA = UΣV0,whereU and V are orthogonal 2 × 2.Σ is diagonal with non-negative diagonal entries.The columns of U are left singular vectors of A.The columns of V are right singular vectors of A.The diagonal entries of Σ are singular values of A.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationSVDDefinitionThe SVD of A is a decompositionA = UΣV0,whereU and V are orthogonal 2 × 2.Σ is diagonal with non-negative diagonal entries.The columns of U are left singular vectors of A.The columns of V are right singular vectors of A.The diagonal entries of Σ are singular values of A.Comment: Let R be a diagonal matrix with unit diagonal entriesThen, if A = UΣV0, then also A = (UR)Σ(RV0).Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationSVDComment: Let R be a diagonal matrix with unit diagonal entriesThen, if A = UΣV0, then also A = (UR)Σ(RV0).Since UR and RV0are still orthogonal, this is another SVD of A.Up to this triviality (i.e., the mulitiplication of the singular vectorsby −1), the SVD is unique whenever Σ(1, 1) > Σ(2, 2).Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivation‘Reverse engineering’ on the SVDAssume that A = UΣV0as before.First fundamental computationAA0= (UΣV0)(UΣV0)0= (UΣV0)(VΣU0) = UΣ(V0V)ΣU0= UΣ2U0,andA0A = VΣ2V0.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivation‘Reverse engineering’ on the SVDAssume that A = UΣV0as before.First fundamental computationAA0= (UΣV0)(UΣV0)0= (UΣV0)(VΣU0) = UΣ(V0V)ΣU0= UΣ2U0,andA0A = VΣ2V0.ConclusionThe left singular vectors are the eigenvectors of AA0.The right singular vectors are the eigenvectors of A0A.The squares of the singular values are the eigenvalues ofA0A and of AA0.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivation‘Reverse engineering’ on the SVDAssume that A = UΣV0as before.Second fundamental computationA0= (UΣV0)0= VΣU0.So,The left singular vectors of A are the right singular vectorsof A0.The left singular vectors of A0are the right singular vectorsof A.The singular values of A and A0are the same.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivation‘Reverse engineering’ on the SVDAssume that A = UΣV0as before.Third fundamental computationAV = UΣ(V0V) = UΣ,i.e., Av1= Σ(1, 1)u1, Av2= Σ(2, 2)u2.Similarly,A0U = VΣ.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationWhat are we really looking for?The SVD reads asAV = UΣ.It therefore find two vector v1, v2(The columns of V) such thatv1, v2is an orthonormal basis for IR2.(Av1, Av2) = 0.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationWhat are we really looking for?The SVD reads asAV = UΣ.It therefore find two vector v1, v2(The columns of V) such thatv1, v2is an orthonormal basis for IR2.(Av1, Av2) = 0.Indeed, if you find such v1, v2, you may defineui:=Avi||Avi||2.Then, u1, u1is an orthonormal basis, too. U is then theirconcatenation.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationWhat are we really looking for?The SVD reads asAV = UΣ.It therefore find two vector v1, v2(The columns of V) such thatv1, v2is an orthonormal basis for IR2.(Av1, Av2) = 0.Indeed, if you find such v1, v2, you may defineui:=Avi||Avi||2.Then, u1, u1is an orthonormal basis, too. U is then theirconcatenation.We get thenAV = UΣ,with V, U as above (orthonormal then), and Σ diagonal, withdiagonal entries ||Avi||2.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationDerviationWe take v1, v2the orthonormal eigenbasis of A0A.We then prove that Av1, Av2are eigenvectors of AA0. If theireigenvalues are different, they must be perpendicular (sinceAA0is symmetric).Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationDerviationWe take v1, v2the orthonormal eigenbasis of A0A.We then prove that Av1, Av2are eigenvectors of AA0. If theireigenvalues are different, they must be perpendicular (sinceAA0is symmetric).So, (λi, vi), is an eigenpair of A0A, and we want Avito be aneigenpair of AA0:(AA0)(Avi) = A(A0A)vi= A(λivi) = λiAvi.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationDerviationWe take v1, v2the orthonormal eigenbasis of A0A.We then prove that Av1, Av2are eigenvectors of AA0. If theireigenvalues are different, they must be perpendicular (sinceAA0is symmetric).So, (λi, vi), is an eigenpair of A0A, and we want Avito be aneigenpair of AA0:(AA0)(Avi) = A(A0A)vi= A(λivi) = λiAvi.The above works if A is non-sigular, and λ16= λ2Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationDemo #3Amos Ron CS513, remote learning