Application of LU factorization Stability of LU Lecture 17 LU factorization continued Amos Ron University of Wisconsin Madison March 19 2021 Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Outline 1 Application of LU factorization Computing determinants Determening Positive de niteness 2 Stability of LU Pivoting Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Blank page Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness Outline 1 Application of LU factorization Computing determinants Determening Positive de niteness 2 Stability of LU Pivoting Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness Determinant of triangular matrices and general matrices Assume A is triangular upper lower then The eigenvalues of A are the diagonal entries of A det A cid 81 m i 1 A i i Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness Determinant of triangular matrices and general matrices Assume A is triangular upper lower then The eigenvalues of A are the diagonal entries of A det A cid 81 m i 1 A i i Computing det A for a general square matrix Option 1 Directly from the de nition Complexity O m hopeless Option 2 Using the multiplication theorem det BC det B det C Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness Determinant of triangular matrices and general matrices Computing det A for a general square matrix Option 1 Directly from the de nition Complexity O m hopeless Option 2 Using the multiplication theorem det BC det B det C Algorithm for computing det A LU factor A det A cid 81 m Complexity O m3 i 1 U i i since det L 1 Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness The three characterizations of positive de niteness Theorem Let A be square symmetric invertible with LDU factorization TFCAE A is SPD A 0 All the main principal minor are positive The diagonal entries of D in A LDU are all positive Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness The three characterizations of positive de niteness Theorem Let A be square symmetric invertible with LDU factorization TFCAE A is SPD A 0 All the main principal minor are positive The diagonal entries of D in A LDU are all positive Proof We only prove that the last condition is equivalent to the rest Since A is symmetric A U cid 48 DU Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness The three characterizations of positive de niteness Proof We only prove that the last condition is equivalent to the rest Since A is symmetric A U cid 48 DU So let v cid 54 0 Av v U cid 48 DU v v DUv Uv Dw w With w Uv Since U is intertible and v cid 54 0 we have w cid 54 0 If D i i 0 i then Dw w 0 easy and we already argued that before Then A is SPD If D i i 0 for some i then we can choose w ei i e we choose v U 1ei Then Dw w Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness The three characterizations of positive de niteness Proof We only prove that the last condition is equivalent to the rest Since A is symmetric A U cid 48 DU So let v cid 54 0 Av v U cid 48 DU v v DUv Uv Dw w With w Uv Since U is intertible and v cid 54 0 we have w cid 54 0 If D i i 0 i then Dw w 0 easy and we already argued that before Then A is SPD If D i i 0 for some i then we can choose w ei i e we choose v U 1ei Then Dei ei D i i 0 and therefore A is not SPD Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Computing determinants Determening Positive de niteness Blank page Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Pivoting Outline 1 Application of LU factorization Computing determinants Determening Positive de niteness 2 Stability of LU Pivoting Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Pivoting A 2 2 examples Consider the matrix A where cid 15 is small When cid 15 0 A is orthogonal hence cond A 1 Therefore for small cid 15 we have cid 19 cid 18 cid 15 1 0 1 cond A 1 Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Pivoting A 2 2 examples Consider the matrix A where cid 15 is small When cid 15 0 A is orthogonal hence cond A 1 Therefore for small cid 15 we have cid 19 cid 18 cid 15 1 0 1 cond A 1 However the LU factorization of A is Then L cid 18 1 cid 19 0 cid 15 1 1 U cid 19 cid 18 cid 15 1 0 cid 15 1 cond L cid 15 1 cond U cid 15 2 So the factorization is unstable Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Pivoting A 2 2 examples What to do The problem is that in the algorithm we de ne v1 2 m A 2 m 1 A 1 1 and the pivot A 1 1 is small compared to other entries of A hence v1 contains large entries This automatically makes L ill conditioned Key Do not use pivots that are small relative to other entries Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Pivoting A 2 2 examples Key Do not use pivots that are small relative to other entries Solution Assuming that we are allowed to shuf e the order of the rows columns we do so Shuf ing both rows and columns full pivoting Shuf ing only rows partial pivoting Amos Ron CS513 remote learning S21 Application of LU factorization Stability of LU Pivoting How to perform the pivoting At each step The input matrix Aj 1 has already undergone some pivoting i e some row and columns were shuf ed In the j cid 48 th step we need to create the vector vj j m Aj 1 j m j Aj 1 j j So we want to bring to the j j location a large entry We can choose a row among the j m rows of Aj 1 And if we do full pivoting we can choose a column from the j m columns of Aj 1 Amos Ron CS513 remote learning S21