MATH 311-504Topics in Applied MathematicsLecture 3-6:The Gram-Schmidt process (continued).Orthogonal systemsLet V be a vector space with an inner product h·, ·iand the induced norm k · k.Definition. A nonempty set S ⊂ V is called anorthogonal system if all vectors in S are mutuallyorthogonal. That is, hx, yi = 0 for any x, y ∈ S,x 6= y.An orthogonal system S ⊂ V is calledorthonormal if kxk = 1 for any x ∈ S.Theorem Any orthogonal system without zerovector is a li nearly independent set.Orthogonal projectionLet V be an inner product space.Let x, v ∈ V , v 6= 0. Then p =hx, vihv, viv is theorthogonal pr ojection of the v ector x onto thevector v. That is, the remainder o = x − p isorthogonal to v.If v1, v2, . . . , vnis an orthogonal set of vectors thenp =hx, v1ihv1, v1iv1+hx, v2ihv2, v2iv2+ ··· +hx, vnihvn, vnivnis the orthogonal projection of the vector x ontothe subspace spanned by v1, . . . , vn. That is, theremainder o = x − p is orthogonal to v1, . . . , vn.The Gram-Schmidt orthogonalization processLet V be a vector space with an inner product.Suppose x1, x2, . . . , xnis a basis f or V . Letv1= x1,v2= x2−hx2, v1ihv1, v1iv1,v3= x3−hx3, v1ihv1, v1iv1−hx3, v2ihv2, v2iv2,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vn= xn−hxn, v1ihv1, v1iv1− ··· −hxn, vn−1ihvn−1, vn−1ivn−1.Then v1, v2, . . . , vnis an orthogonal basis for V .Any basisx1, x2, . . . , xn−→Orthogonal basisv1, v2, . . . , vnProperties of the Gram-Schmidt process :• vk= xk− (α1x1+ ··· + αk −1xk −1), 1 ≤ k ≤ n;• the span of v1, . . . , vkis the same as the spanof x1, . . . , xk;• vkis orthogonal to x1, . . . , xk −1;• vk= xk− pk, where pkis the orthogonalpro jection of the vector xkon the subs pace spannedby x1, . . . , xk −1;• kvkk is the distance f rom xkto the subspacespanned by x1, . . . , xk −1.Problem. Find the distance from the pointy = (0, 0, 0, 1) to the subspace Π ⊂ R4spanned byvectors x1= (1, −1, 1, −1 ), x2= (1, 1, 3, −1 ), andx3= (−3, 7, 1, 3 ).Let us apply the Gram -Schmidt process to vectorsx1, x2, x3, y. We should obtain an orthogonalsystem v1, v2, v3, v4. The desired distance will be|v4|.x1= (1, −1, 1, −1 ), x2= (1, 1, 3, −1),x3= (−3, 7, 1, 3 ), y = (0, 0, 0, 1).v1= x1= (1, −1, 1, −1 ),v2= x2−hx2, v1ihv1, v1iv1= (1, 1, 3, −1 )−44(1, −1, 1, −1)= (0, 2, 2, 0 ),v3= x3−hx3, v1ihv1, v1iv1−hx3, v2ihv2, v2iv2= (−3, 7, 1, 3 ) −−124(1, −1, 1, −1) −168(0, 2, 2, 0)= (0, 0, 0, 0 ).The Gram-Schmidt process can be used to checklinear independence o f vector s !The vector x3is a linear combination of x1and x2.Π is a plane, not a 3-dimensional subspace.We sho uld orthogonalize vector s x1, x2, y.v4= y −hy, v1ihv1, v1iv1−hy, v2ihv2, v2iv2= (0, 0, 0, 1 ) −−14(1, −1, 1, −1) −08(0, 2, 2, 0)= (1/4, −1/4, 1 /4, 3/4).|v4| =14, −14,14,34=14|(1, −1, 1, 3)| =√124=√32.Problem. Find the distance from the pointz = (0, 0, 1, 0) to the plane Π that passes throughthe poi nt x0= (1, 0, 0, 0) and is par allel to thevectors v1= (1, −1, 1, −1 ) and v2= (0, 2, 2, 0 ).The plane Π is not a subspace of R4as it does notpass through the or igin. Let Π0= Span(v1, v2).Then Π = Π0+ x0.Hence the distance from the point z to the plane Πis the same as the distance from the point z − x0to the pl ane Π0.We shall apply the Gram-Schmidt pr ocess to vectorsv1, v2, z − x0. This will yield an orthogonal sys temw1, w2, w3. The desi red distance wil l be |w3|.v1= (1, −1, 1, −1 ), v2= (0, 2, 2, 0), z − x0= ( −1, 0, 1, 0).w1= v1= (1, −1, 1, −1 ),w2= v2−hv2, w1ihw1, w1iw1= v2= (0, 2, 2, 0 ) as v2⊥ v1.w3= (z − x0) −hz − x0, w1ihw1, w1iw1−hz − x0, w2ihw2, w2iw2= (−1, 0, 1, 0 ) −04(1, −1, 1, −1) −28(0, 2, 2, 0)= (−1, −1/2, 1/2, 0).|w3| =−1, −12,12, 0=12|(−2, −1, 1, 0)| =√62=r32.Modifications of the Gram - Schmidt processThe first modi fication combines orthogonalizationwith normalization. Suppose x1, x2, . . . , xnis abasis for an inner product space V . Letv1= x1, w1=v1kv1k,v2= x2− hx2, w1iw1, w2=v2kv2k,v3= x3− hx3, w1iw1− hx3, w2iw2, w3=v3kv3k,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vn= xn− hxn, w1iw1− ··· −hxn, wn−1iwn−1,wn=vnkvnk.Then w1, w2, . . . , wnis an orthonormal basis for V .Modifications of the Gram - Schmidt processFurther modification is a r ecur s ive pro ces s which ismore stable to roundoff errors than the originalprocess. Suppose x1, x2, . . . , xnis a basis f or aninner pr oduct space V . Letw1=x1kx1k,x′2= x2− hx2, w1iw1,x′3= x3− hx3, w1iw1,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .x′n= xn− hxn, w1iw1.Then w1, x′2, . . . , x′nis a basis f or V , k w1k = 1,and w1is orthogonal to x′2, . . . , x′n. Now repeat theprocess with vectors x′2, . . . , x′n, and so on.Problem. Approximate the functio n f (x) = exonthe interval [−1, 1] by a quadratic polynomial.The best approximation would be a polynomial p(x)that minimizes the distance relative to the uniformnor m:kf − pk∞= max|x |≤1|f (x) − p(x)|.However there is no analy tic way to find such apolynomial. Instead, we ar e going to find a “leastsquares” approximation that minimizes the integralnor mkf − pk2=Z1−1|f (x) − p(x)|2dx1/2.The norm k · k2is induced by the inner producthg, hi =Z1−1g(x)h(x) dx.Therefore kf − pk2is minimal if p is theorthogonal projection of the f uncti on f on thesubspace P2of quadratic polynom ials.We sho uld apply the Gram-Schmidt process to thepolynomials 1, x, x2which form a basis f or P2.This would yield an orthogonal basis p0, p1, p2.Thenp(x) =hf , p0ihp0, p0ip0(x) +hf , p1ihp1, p1ip1(x) +hf , p2ihp2,