UW-Madison CS 513 - Lecture 4: Introduction

Matrix NormsCharacterizing the -normCharacterizing the 2-normPositive definite matricesDefinition and exampleMatrix NormsPositive definite matricesLecture 4: IntroductionAmos RonUniversity of Wisconsin - MadisonFebruary 01, 2021Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesOutline1Matrix NormsCharacterizing the ∞-normCharacterizing the 2-norm2Positive definite matricesDefinition and exampleAmos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normOutline1Matrix NormsCharacterizing the ∞-normCharacterizing the 2-norm2Positive definite matricesDefinition and exampleAmos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normBlank pageAmos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-norm∞-normTheorem: Computing the ∞-norm1For an A m × n,||A||1= ||A0||∞.2Let b01, . . . , b0mbe the rows of A. Then||A||∞= max1≤i≤m||bi||1.Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-norm∞-normTheorem: Computing the ∞-norm1For an A m × n,||A||1= ||A0||∞.2Let b01, . . . , b0mbe the rows of A. Then||A||∞= max1≤i≤m||bi||1.Comment: The equivalence of the two conditions above followsdirectly from the characterization of the 1-norm.Comment: Assertion (2) above can be proved directly, using asimilar approach (but with different details) to the prooof of the1-norm case.Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-norm∞-normWe show how to prove ||A||1= ||A0||∞directly from basic LinearAlgebra principles.Step I: Show that, for any v ∈ Rm,||v||1= max{(v, w) : ||w||∞= 1}, and||v||∞= max{(v, w) : ||w||1= 1}.Step II: Since ||A||1= max{||Av||1: ||v||1= 1}, it follows that||A||1= max{(Av, w) : ||v||1= 1, ||w||∞= 1}.Step III: Since ||A0||∞= max{||A0w||∞: ||w||∞= 1}, it followsthat||A0||∞= max{(A0w, v) : ||v||1= 1, ||w||∞= 1}.Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normThe 2-norm of a matrixSome basics:||v||22= (v, v).Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normThe 2-norm of a matrixSome basics:||v||22= (v, v).Whatever A is, A0A is symmetric, and its eigenvalues arenon-negative.(BC)0= C0B0=⇒ (A0A)0= A0A00= A0A.Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normThe 2-norm of a matrixSome basics:||v||22= (v, v).Whatever A is, A0A is symmetric, and its eigenvalues arenon-negative.(A0A)v = λv =⇒ λ||v||22= (λv, v) = (A0Av, v) = (Av, Av) = ||Av||22,=⇒ λ =||Av||22||v||22≥ 0.Also:λ ≤ ||A||22.Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normThe 2-norm of a matrixWhatever A is, A0A is symmetric, and its eigenvalues arenon-negative.DefinitionA right singular vector of A is an eigenvector of A0A.An s ≥ 0 is a singular value of A is s2∈ σ(A0A).Notation (spectral radius): A square:ρ(A) := max{|λ| : λ ∈ σ(A)}.So:||A||2≥pρ(A0A).Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normCharacterizing the 2-normTheorem: Chracterizing the 2-norm||A||2=pρ(A0A),i.e., ||A||2= the largest singular value of A.Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normCharacterizing the 2-normTheorem: Chracterizing the 2-norm||A||2=pρ(A0A),i.e., ||A||2= the largest singular value of A.Proof: We already saw that ||A||2≥p(ρ(A0A)).Now, Let v ∈ Rm, such that ||v||2= 1, and ||A||2= ||Av||2. LetA0A = QDQ0be the Schur decomposition of A0A. Then||A||22= ||Av||22= (Av, Av) = (A0Av, v) =(QDQ0v, v) = (DQ0v, Q0v).Denote w := Q0v. Since Q0is orthogonal, ||w||2= ||v||2= 1.Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normCharacterizing the 2-norm||A||22= ||Av||22= (Av, Av) = (A0Av, v) =(QDQ0v, v) = (DQ0v, Q0v).Denote w := Q0v. Since Q0is orthogonal, ||w||2= ||v||2= 1.So,||A||22= (Dw, w) =mXi=1D(i, i)w(i)2≤mXi=1ρ(A0A)w(i)2= ρ(A0A)mXi=1w(i)2= ρ(A0A).So, ||A||2≤pρ(A0A).Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesCharacterizing the ∞-normCharacterizing the 2-normDemo #2Amos Ron CS513, remote learning S21Matrix NormsPositive definite matricesDefinition and exampleOutline1Matrix NormsCharacterizing the ∞-normCharacterizing the 2-norm2Positive definite matricesDefinition and exampleAmos Ron CS513, remote learning S21Matrix NormsPositive definite matricesDefinition and exampleDefinition of Positive DefinitenessAmos Ron CS513, remote learning