MATH 304Linear AlgebraLecture 28:Orthogonal bases.The Gram-Schmidt orthogonalization process.Orthogonal setsLet V be an inner product space with an innerproduct h·, ·i and the induced norm kvk =phv, vi.Definition. A nonempty set S ⊂ V of nonzerovectors is called an orthogonal set if all vectors inS are mutually orthogonal. That is, 0 /∈ S andhx, yi = 0 for any x, y ∈ S, x 6= y.An orthogonal set S ⊂ V is called orthonormal ifkxk = 1 for any x ∈ S.Remark. Vectors v1, v2, . . . , vk∈ V form anorthonormal set if and only ifhvi, vji =1 if i = j0 if i 6= jExamples. • V = Rn, hx, yi = x · y.The standard basis e1= (1, 0, 0, . . . , 0),e2= (0, 1, 0, . . . , 0), . . . , en= (0, 0, 0, . . . , 1).It is an orthonormal set.• V = R3, hx, yi = x · y.v1= (3, 5, 4), v2= (3, −5, 4), v3= (4, 0, −3).v1· v2= 0, v1· v3= 0, v2· v3= 0,v1· v1= 50, v2· v2= 50, v3· v3= 25.Thus the set {v1, v2, v3} is orthogonal but notorthonormal. An orthonormal set is formed bynormalized vectors w1=v1kv1k, w2=v2kv2k,w3=v3kv3k.• V = C [−π, π], hf , g i =Zπ−πf (x)g(x) dx.f1(x) = sin x, f2(x) = sin 2x, . . . , fn(x) = sin nx, . . .hfm, fni =Zπ−πsin(mx) sin(nx) dx =π if m = n0 if m 6= nThus the set {f1, f2, f3, . . . } is orthogonal but notorthonormal.It is orthonormal with respect to a scaled innerproducthhf , gii =1πZπ−πf (x)g(x) dx.Orthogonality =⇒ linear independenceTheorem Suppose v1, v2, . . . , vkare nonzerovectors that form an orthogonal set. Thenv1, v2, . . . , vkare linearly independent.Proof: Suppose t1v1+ t2v2+ · · · + tkvk= 0for some t1, t2, . . . , tk∈ R.Then for any index 1 ≤ i ≤ k we haveht1v1+ t2v2+ · · · + tkvk, vii = h0, vii = 0.=⇒ t1hv1, vii + t2hv2, vii + · · · + tkhvk, vii = 0By orthogonality, tihvi, vii = 0 =⇒ ti= 0.Orthonormal basesLet v1, v2, . . . , vnbe an orthonormal basis for aninner product space V .Theorem Let x = x1v1+ x2v2+ · · · + xnvnandy = y1v1+ y2v2+ · · · + ynvn, where xi, yj∈ R. Then(i) hx, yi = x1y1+ x2y2+ · · · + xnyn,(ii) kxk =px21+ x22+ · · · + x2n.Proof: (ii) follows from (i) when y = x.hx, yi =*nXi=1xivi,nXj=1yjvj+=nXi=1xi*vi,nXj=1yjvj+=nXi=1nXj=1xiyjhvi, vji =nXi=1xiyi.Orthogonal projectionTheorem Let V be an inner product space and V0be a finite-dimensional subspace of V . Then anyvector x ∈ V is uniquely represented as x = p + o,where p ∈ V0and o ⊥ V0.The component p is the orthogonal projection ofthe vector x onto the subspace V0. We havekok = kx − pk = minv∈V0kx − vk.That is, the distance from x to the subspace V0iskok.V0opxLet V be an inner product space. Let p be theorthogonal projection of a vector x ∈ V onto afinite-dimensional subspace V0.If V0is a one-dimensional subspace spanned by avector v then p =hx, vihv, viv.If v1, v2, . . . , vnis an orthogonal basis for V0thenp =hx, v1ihv1, v1iv1+hx, v2ihv2, v2iv2+ · · · +hx, vnihvn, vnivn.Indeed, hp, vii =nXj=1hx, vjihvj, vjihvj, vii =hx, viihvi, viihvi, vii = hx, vii=⇒ hx−p, vii = 0 =⇒ x−p ⊥ vi=⇒ x−p ⊥ V0.The Gram-Schmidt orthogonalization processLet V be a vector space with an inner product.Suppose x1, x2, . . . , xnis a basis for V . Letv1= x1,v2= x2−hx2, v1ihv1, v1iv1,v3= x3−hx3, v1ihv1, v1iv1−hx3, v2ihv2, v2iv2,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vn= xn−hxn, v1ihv1, v1iv1− · · · −hxn, vn−1ihvn−1, vn−1ivn−1.Then v1, v2, . . . , vnis an orthogonal basis for V .Span(v1, v2) = Span(x1, x2)v3p3x3Any basisx1, x2, . . . , xn−→Orthogonal basisv1, v2, . . . , vnProperties of the Gram-Schmidt process:• vk= xk− (α1x1+ · · · + αk−1xk−1), 1 ≤ k ≤ n;• the span of v1, . . . , vkis the same as the spanof x1, . . . , xk;• vkis orthogonal to x1, . . . , xk−1;• vk= xk− pk, where pkis the orthogonalprojection of the vector xkon the subspace spannedby x1, . . . , xk−1;• kvkk is the distance from xkto the subspacespanned by x1, . . . , xk−1.NormalizationLet V be a vector space with an inner product.Suppose v1, v2, . . . , vnis an orthogonal basis for V .Let w1=v1kv1k, w2=v2kv2k,. . . , wn=vnkvnk.Then w1, w2, . . . , wnis an orthonormal basis for V .Theorem Any finite-dimensional vector space withan inner product has an orthonormal basis.Remark. An infin ite-dimensional vector space withan inner product may or may not have anorthonormal b asis.Orthogonalization / NormalizationAn alternative form of the Gram-Schmidt process combinesorthogonalization with normalization.Suppose x1, x2, . . . , xnis a basis for an innerproduct space V . Letv1= x1, w1=v1kv1k,v2= x2− hx2, w1iw1, w2=v2kv2k,v3= x3− hx3, w1iw1− hx3, w2iw2, w3=v3kv3k,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vn= xn− hxn, w1iw1− · · · − hxn, wn−1iwn−1,wn=vnkvnk.Then w1, w2, . . . , wnis an orthonormal basis for V
View Full Document
Unlocking...