New version page

# UW-Madison CS 513 - Lecture 5: Introduction

Pages: 18
Documents in this Course

17 pages

24 pages

13 pages

15 pages

2 pages

2 pages

2 pages

3 pages

2 pages

Load more
Upgrade to remove ads

## This preview shows page 1-2-3-4-5-6 out of 18 pages.

View Full Document
Premium Document
Do you want full access? Go Premium and unlock all 18 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 18 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 18 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 18 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 18 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 18 pages.
Access to all documents
Download any document
Ad free experience

Upgrade to remove ads
Unformatted text preview:

The Singular Value DecompositionDefining the SVD, 22 caseDerivationThe Singular Value DecompositionLecture 5: IntroductionAmos RonUniversity of Wisconsin - MadisonFebruary 03, 2021Amos Ron CS513, remote learning S21The Singular Value DecompositionOutline1The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationAmos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationOutline1The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationAmos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationBlank pageAmos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationSVDA is any matrix 2 × 2.DefinitionThe SVD of A is a decompositionA = UΣV0,whereU and V are orthogonal 2 × 2.Σ is diagonal with non-negative diagonal entries.The columns of U are left singular vectors of A.The columns of V are right singular vectors of A.The diagonal entries of Σ are singular values of A.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationSVDDefinitionThe SVD of A is a decompositionA = UΣV0,whereU and V are orthogonal 2 × 2.Σ is diagonal with non-negative diagonal entries.The columns of U are left singular vectors of A.The columns of V are right singular vectors of A.The diagonal entries of Σ are singular values of A.Comment: Let R be a diagonal matrix with unit diagonal entriesThen, if A = UΣV0, then also A = (UR)Σ(RV0).Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationSVDComment: Let R be a diagonal matrix with unit diagonal entriesThen, if A = UΣV0, then also A = (UR)Σ(RV0).Since UR and RV0are still orthogonal, this is another SVD of A.Up to this triviality (i.e., the mulitiplication of the singular vectorsby −1), the SVD is unique whenever Σ(1, 1) > Σ(2, 2).Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivation‘Reverse engineering’ on the SVDAssume that A = UΣV0as before.First fundamental computationAA0= (UΣV0)(UΣV0)0= (UΣV0)(VΣU0) = UΣ(V0V)ΣU0= UΣ2U0,andA0A = VΣ2V0.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivation‘Reverse engineering’ on the SVDAssume that A = UΣV0as before.First fundamental computationAA0= (UΣV0)(UΣV0)0= (UΣV0)(VΣU0) = UΣ(V0V)ΣU0= UΣ2U0,andA0A = VΣ2V0.ConclusionThe left singular vectors are the eigenvectors of AA0.The right singular vectors are the eigenvectors of A0A.The squares of the singular values are the eigenvalues ofA0A and of AA0.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivation‘Reverse engineering’ on the SVDAssume that A = UΣV0as before.Second fundamental computationA0= (UΣV0)0= VΣU0.So,The left singular vectors of A are the right singular vectorsof A0.The left singular vectors of A0are the right singular vectorsof A.The singular values of A and A0are the same.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivation‘Reverse engineering’ on the SVDAssume that A = UΣV0as before.Third fundamental computationAV = UΣ(V0V) = UΣ,i.e., Av1= Σ(1, 1)u1, Av2= Σ(2, 2)u2.Similarly,A0U = VΣ.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationWhat are we really looking for?The SVD reads asAV = UΣ.It therefore find two vector v1, v2(The columns of V) such thatv1, v2is an orthonormal basis for IR2.(Av1, Av2) = 0.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationWhat are we really looking for?The SVD reads asAV = UΣ.It therefore find two vector v1, v2(The columns of V) such thatv1, v2is an orthonormal basis for IR2.(Av1, Av2) = 0.Indeed, if you find such v1, v2, you may defineui:=Avi||Avi||2.Then, u1, u1is an orthonormal basis, too. U is then theirconcatenation.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationWhat are we really looking for?The SVD reads asAV = UΣ.It therefore find two vector v1, v2(The columns of V) such thatv1, v2is an orthonormal basis for IR2.(Av1, Av2) = 0.Indeed, if you find such v1, v2, you may defineui:=Avi||Avi||2.Then, u1, u1is an orthonormal basis, too. U is then theirconcatenation.We get thenAV = UΣ,with V, U as above (orthonormal then), and Σ diagonal, withdiagonal entries ||Avi||2.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationDerviationWe take v1, v2the orthonormal eigenbasis of A0A.We then prove that Av1, Av2are eigenvectors of AA0. If theireigenvalues are different, they must be perpendicular (sinceAA0is symmetric).Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationDerviationWe take v1, v2the orthonormal eigenbasis of A0A.We then prove that Av1, Av2are eigenvectors of AA0. If theireigenvalues are different, they must be perpendicular (sinceAA0is symmetric).So, (λi, vi), is an eigenpair of A0A, and we want Avito be aneigenpair of AA0:(AA0)(Avi) = A(A0A)vi= A(λivi) = λiAvi.Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationDerviationWe take v1, v2the orthonormal eigenbasis of A0A.We then prove that Av1, Av2are eigenvectors of AA0. If theireigenvalues are different, they must be perpendicular (sinceAA0is symmetric).So, (λi, vi), is an eigenpair of A0A, and we want Avito be aneigenpair of AA0:(AA0)(Avi) = A(A0A)vi= A(λivi) = λiAvi.The above works if A is non-sigular, and λ16= λ2Amos Ron CS513, remote learning S21The Singular Value DecompositionDefining the SVD, 2 × 2 caseDerivationDemo #3Amos Ron CS513, remote learning

View Full Document
Download Lecture 5: Introduction
Our administrator received your request to download this document. We will send you the file to your email shortly.
Unlocking...
Sign Up

Join to view Lecture 5: Introduction 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?