MATH 304Linear AlgebraLecture 28:Orthogonal bases.The Gram-Schmidt orthogonalization process.Orthogonal setsLet V be an inner product space with an innerproduct h·, ·i and the induced norm kvk =phv, vi.Definition. Nonzero vectors v1, v2, . . . , vk∈ Vform an orthogonal set if they are orthogonal toeach other: hvi, vji = 0 for i 6= j.If, in addition, all vectors are of unit norm,kvik = 1, then v1, v2, . . . , vkis called anorthonormal set.Theorem Any orthogonal set is linearlyindependent.Orthonormal basesLet v1, v2, . . . , vnbe an orthonormal basis for aninner product space V .Theorem Let x = x1v1+ x2v2+ · · · + xnvnandy = y1v1+ y2v2+ · · · + ynvn, where xi, yj∈ R. Then(i) hx, yi = x1y1+ x2y2+ · · · + xnyn,(ii) kxk =px21+ x22+ · · · + x2n.Proof: (ii) follows from (i) when y = x.hx, yi =*nXi=1xivi,nXj=1yjvj+=nXi=1xi*vi,nXj=1yjvj+=nXi=1nXj=1xiyjhvi, vji =nXi=1xiyi.Orthogonal projectionTheorem Let V be an inner product space and V0be a finite-dimensional subspace of V . Then anyvector x ∈ V is uniquely represented as x = p + o,where p ∈ V0and o ⊥ V0.The component p is the orthogonal projection ofthe vector x onto the subspace V0. We havekok = kx − pk = minv∈V0kx − vk.That is, the distance from x to the subspace V0iskok.V0opxLet V be an inner product space. Let p be theorthogonal projection of a vector x ∈ V onto afinite-dimensional subspace V0.If V0is a one-dimensional subspace spanned by avector v then p =hx, vihv, viv.If v1, v2, . . . , vnis an orthogonal basis for V0thenp =hx, v1ihv1, v1iv1+hx, v2ihv2, v2iv2+ · · · +hx, vnihvn, vnivn.Indeed, hp, vii =nXj=1hx, vjihvj, vjihvj, vii =hx, viihvi, viihvi, vii = hx, vii=⇒ hx−p, vii = 0 =⇒ x−p ⊥ vi=⇒ x−p ⊥ V0.The Gram-Schmidt orthogonalization processLet V be a vector space with an inner product.Suppose x1, x2, . . . , xnis a basis for V . Letv1= x1,v2= x2−hx2, v1ihv1, v1iv1,v3= x3−hx3, v1ihv1, v1iv1−hx3, v2ihv2, v2iv2,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vn= xn−hxn, v1ihv1, v1iv1− · · · −hxn, vn−1ihvn−1, vn−1ivn−1.Then v1, v2, . . . , vnis an orthogonal basis for V .Span(v1, v2) = Span(x1, x2)v3p3x3Any basisx1, x2, . . . , xn−→Orthogonal basisv1, v2, . . . , vnProperties of the Gram-Schmidt process:• vk= xk− (α1x1+ · · · + αk−1xk−1), 1 ≤ k ≤ n;• the span of v1, . . . , vkis the same as the spanof x1, . . . , xk;• vkis orthogonal to x1, . . . , xk−1;• vk= xk− pk, where pkis the orthogonalprojection of the vector xkon the subspace spannedby x1, . . . , xk−1;• kvkk is the distance from xkto the subspacespanned by x1, . . . , xk−1.NormalizationLet V be a vector space with an inner product.Suppose v1, v2, . . . , vnis an orthogonal basis for V .Let w1=v1kv1k, w2=v2kv2k,. . . , wn=vnkvnk.Then w1, w2, . . . , wnis an orthonormal basis for V .Theorem Any finite-dimensional vector space withan inner product has an orthonormal basis.Remark. An infinite-dimensional vector space withan inner product may or may not have anorthonormal basis.Orthogonalization / NormalizationAn alternative form of the Gram-Schmidt process combinesorthogonalization with normalization.Suppose x1, x2, . . . , xnis a basis for an innerproduct space V . Letv1= x1, w1=v1kv1k,v2= x2− hx2, w1iw1, w2=v2kv2k,v3= x3− hx3, w1iw1− hx3, w2iw2, w3=v3kv3k,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vn= xn− hxn, w1iw1− · · · − hxn, wn−1iwn−1,wn=vnkvnk.Then w1, w2, . . . , wnis an orthonormal basis for V .Problem. Let V0be a subspace of dimension k inRn. Let x1, x2, . . . , xkbe a basis for V0.(i) Find an orthogonal basis for V0.(ii) Extend it to an orthogonal basis for Rn.Approach 1. Extend x1, . . . , xkto a basis x1, x2, . . . , xnforRn. Then apply the Gram-Schmidt process to the extendedbasis. We shall obtain an orthogonal basis v1, . . . , vnfor Rn.By construction, Span(v1, . . . , vk) = Span(x1, . . . , xk) = V0.It follows that v1, . . . , vkis a basis for V0. Clearly, it isorthogonal.Approach 2. First apply the Gram-Schmidt process tox1, . . . , xkand obtain an orthogonal basis v1, . . . , vkfor V0.Secondly, find a basis y1, . . . , ymfor the orthogonalcomplement V⊥0and apply the Gram-Schmidt process to itobtaining an orthogonal basis u1, . . . , umfor V⊥0. Thenv1, . . . , vk, u1, . . . , umis an orthogonal basis for Rn.Problem. Let Π be the plane in R3spanned byvectors x1= (1, 2, 2) and x2= (−1, 0, 2).(i) Find an orthonormal basis for Π.(ii) Extend it to an orthonormal basis for R3.x1, x2is a basis for the plane Π. We can extend itto a basis for R3by adding one vector from thestandard basis. For instance, vectors x1, x2, andx3= (0, 0, 1) form a basis for R3because1 2 2−1 0 20 0 1=1 2−1 0= 2 6= 0.Using the Gram-Schmidt process, we orthogonalizethe basis x1= (1, 2, 2), x2= (−1, 0, 2), x3= (0, 0, 1):v1= x1= (1, 2, 2),v2= x2−hx2, v1ihv1, v1iv1= (−1, 0, 2) −39(1, 2, 2)= (−4/3, −2/3, 4/3),v3= x3−hx3, v1ihv1, v1iv1−hx3, v2ihv2, v2iv2= (0, 0, 1) −29(1, 2, 2) −4/34(−4/3, −2/3, 4/3)= (2/9, −2/9, 1/9).Now v1= (1, 2, 2), v2= (−4/3, −2/3, 4/3),v3= (2/9, −2/9, 1/9) is an orthogonal basis for R3while v1, v2is an orthogonal basis for Π. It remainsto normalize these vectors.hv1, v1i = 9 =⇒ kv1k = 3hv2, v2i = 4 =⇒ kv2k = 2hv3, v3i = 1/9 =⇒ kv3k = 1/3w1= v1/kv1k = (1/3, 2/3, 2/3) =13(1, 2, 2),w2= v2/kv2k = (−2/3, −1/3, 2/3) =13(−2, −1, 2),w3= v3/kv3k = (2/3, −2/3, 1/3) =13(2, −2, 1).w1, w2is an orthonormal basis for Π.w1, w2, w3is an orthonormal basis for
View Full Document