• We can deal with “complicated” linear systems, but what to do if there is no solutionsand we want a “best” approximate solution?This is important for ma ny applications, including fitting data.• Suppos e Ax = b has no solution. This means b is not in Col(A).Idea: find “best” approximate solution by replacingb with its projection onto Col(A).• Recall: if v1,, vnare (pairwise) orthogonal:v1·(c1v1++ cnvn) = c1v1·v1Implies: the v1,, vnare independent (unless one is the zero vector)Orthogonal basesDefinition 1. A basis v1,, vnof a vector space V is an orthogonal basis if thevectors are (pairwise) orthogonal.Example 2. The standard basis100,010,001is an orthogonal basis for R3.Example 3. Are the vectors1−10,110,001an orthogonal basis for R3?Solution.Armin [email protected]1−10·110= 01−10·001= 0110·001= 0So this is an orthogonal basis.Note that we do not need to check that the three vectors are independent. That followsfrom their orthogonality.Example 4. Suppos e v1,, vnis an orthogonal basis of V , and th a t w is in V . Findc1,, cnsuch thatw = c1v1++ cnvn.Solution. Take the dot product o f v1with both sides:v1·w = v1·(c1v1++ cnvn)= c1v1·v1+ c2v1·v2++ cnv1·vn= c1v1·v1Hence, c1=v1·wv1·v1. In general, cj=vj·wvj·vj.If v1,, vnis an orthogonal basis of V , and w is in V , thenw = c1v1++ cnvnwith cj=w ·vjvj·vj.Example 5. Express374in terms of the basis1−10,110,001.Solution.374= c11−10+ c2110+ c3001=374·1−101−10·1−101−10+374·110110·110110+374·001001·001001=−421−10+102110+41001Armin [email protected] 6. A basis v1,, vnof a vector space V is an orthonormal basis if thevectors are orthogonal and have len g th1.Example 7. The standard basis100,010,001is an orthonormal basis for R3.If v1,, vnis an orthonormal basis of V , and w is in V , thenw = c1v1++ cnvnwith cj= vj·w.Example 8. Express374in terms of the basis100,010,001.Solution. That’s trivial, of course:374= 3100+ 7010+ 4001But note that the coeffi cients are374·100= 3,374·010= 7,374·001= 4.Example 9. Is the basis1−10,110,001orthonormal? If not, normalize the vectorsto produce an orthonormal basis.Solution.1−10has length1−10·1−10s= 2√normalized:12√1−10110has length110·110s= 2√normalized:12√110001has length001·001s= 1is already normalized:001The corresponding orthonormal basis is12√1−10,12√110,001.Armin [email protected] 10. Express374in terms of the basis12√1−10,12√110,001.Solution.374·12√1−10=−42√,374·12√110=102√,374·001= 4.Hence, just as in Example 5:374=−42√12√1−10+102√12√110+ 4001Orthogonal projectionsyxˆxx⊥Definition 11. The orthog on al projection of vector x ontovector y isxˆ=x · yy · yy.• The vector xˆis the cl osest vector to x, which i s inspan{y }.• Characterized by: the “error” x⊥= x −xˆis orthogonal tospan{y }.• To find the formula for xˆ, start with xˆ= cy.(x −xˆ) · y = (x −cy) · y = x ·y −cy · y@wanted0It follows tha t c =x · yy · y.x⊥is also called the co mponent of x orthogonal to y.Example 12. What is the orthogonal projection of x =−84onto y =31?Solution.xˆ=x · yy · yy =−8 ·3 + 4 ·132+ 1231= −231=−6−2The componen t of x orthogonal to y isx −xˆ=−84−−6−2=−26.(Note that, indeed−26and31are orthogonal.)Armin [email protected] 13. What are the orthogonal projections of211onto each of the vectors1−10,110,001?Solution.211on1−10:2 ·1 + 1 ·(−1) + 1 ·012+ (−1)2+ 021−10=121−10211on110:2 · 1 + 1 ·1 + 1 ·012+ 12+ 02110=32110211on001:2 · 0 + 1 ·0 + 1 ·102+ 02+ 12001=001Note that these sum up to121−10+32110+001=211!That’s because the three vectors are an orthog onal basis forR3.Recall: If v1,, vnis an orthogonal basis of V , and w is in V , thenw = c1v1++ cnvnwith cj=w ·vjvj·vj. wdecomposes as the sum of its pr ojections onto each basis vectorArmin
View Full Document
Unlocking...