MATH 304Linear AlgebraLecture 19:Inner product spaces.Orthogonal sets.The Gram-Schmidt process.NormThe notion of norm generalizes the notion of lengthof a vector in Rn.Definition. Let V be a vector space. A functionα : V → R is called a norm on V if it has thefollowing properties:(i) α(x) ≥ 0, α(x) = 0 only for x = 0 (positivity)(ii) α(rx) = |r | α(x) for all r ∈ R (homogeneity)(iii) α(x + y) ≤ α(x) + α(y) (triangle inequality)Notation. The norm of a vector x ∈ V is usuallydenoted kxk. Different norms on V aredistinguished by subscripts, e.g., kxk1and kxk2.Examples. V = Rn, x = (x1, x2, . . . , xn) ∈ Rn.• kxk∞= max(|x1|, |x2|, . . . , |xn|).• kxkp=|x1|p+ |x2|p+ · · · + |xn|p1/p, p ≥ 1.Examples. V = C [a, b], f : [a, b] → R.• kf k∞= maxa≤x≤b|f (x)|.• kf kp=Zba|f (x)|pdx1/p, p ≥ 1.Normed vector spaceDefinition. A normed vector space is a vectorspace endowed with a norm.The norm defin es a distance function on the normedvector space: dist(x, y) = kx − yk.Then we say that a sequence x1, x2, . . . convergesto a vector x if dist(x, xn) → 0 as n → ∞.Also, we say that a vector x is a goodapproximation of a vector x0if dist(x, x0) is small.Inner productThe notion of inner product generalizes the notionof dot product of vectors in Rn.Definition. Let V be a vector space. A functionβ : V × V → R, usually denoted β(x, y) = hx, yi,is called an inner product on V if it is positive,symmetric, and bilinear. That is, if(i) hx, xi ≥ 0, hx, xi = 0 only for x = 0 (positivity)(ii) hx, yi = hy, xi (symmetry)(iii) hrx, yi = r hx, yi (homogeneity)(iv) hx + y, zi = hx, zi + hy, zi (distributive law)An inner product space is a vector space end owedwith an inner product.Examples. V = Rn.• hx, yi = x · y = x1y1+ x2y2+ · · · + xnyn.• hx, yi = d1x1y1+ d2x2y2+ · · · + dnxnyn,where d1, d2, . . . , dn> 0.• hx, yi = (Dx) · (Dy),where D is an invertible n×n matrix.Remarks. (a) Invertibility of D is necessary to showthat hx, xi = 0 =⇒ x = 0.(b) The second example is a particular case of thethird one when D = diag(d1/21, d1/22, . . . , d1/2n).Problem. Find an inner product on R2such thathe1, e1i = 2, he2, e2i = 3, and he1, e2i = −1,where e1= (1, 0), e2= (0, 1).Let x = (x1, x2), y = (y1, y2) ∈ R2.Then x = x1e1+ x2e2, y = y1e1+ y2e2.Using bilinearity, we obtainhx, yi = hx1e1+ x2e2, y1e1+ y2e2i= x1he1, y1e1+ y2e2i + x2he2, y1e1+ y2e2i= x1y1he1, e1i + x1y2he1, e2i + x2y1he2, e1i + x2y2he2, e2i= 2x1y1− x1y2− x2y1+ 3x2y2.Examples. V = C [a, b].• hf , gi =Zbaf (x)g (x) dx.• hf , gi =Zbaf (x)g (x)w(x) dx,where w is bounded, piecewise continuous, andw > 0 everywhere on [a, b].w is called the weight function.Theorem Suppose hx, yi is an inner product on avector space V . Thenhx, yi2≤ hx, xihy, yi for all x, y ∈ V .Proof: For any t ∈ R let vt= x + ty. T henhvt, vti = hx, xi + 2thx, yi + t2hy, yi.The right-hand side is a quadratic polynomial in t(provided that y 6= 0). Since hvt, vti ≥ 0 for all t,the discriminant D is nonpositive. ButD = 4hx, yi2− 4hx, xihy, yi.Cauchy-Schwarz Inequality:|hx, yi| ≤phx, xiphy, yi.Cauchy-Schwarz Inequality:|hx, yi| ≤phx, xiphy, yi.Corollary 1 |x · y| ≤ |x| |y| for all x, y ∈ Rn.Equivalently, for all xi, yi∈ R,(x1y1+ · · · + xnyn)2≤ (x21+ · · · + x2n)(y21+ · · · + y2n).Corollary 2 For any f , g ∈ C [a, b],Zbaf (x)g(x) dx2≤Zba|f (x)|2dx ·Zba|g(x)|2dx.Norms induced by inner productsTheorem Suppose hx, yi is an inner product on avector space V . Then kxk =phx, xi is a norm.Proof: Positivity is obvious. Homogeneity:krxk =phrx, r xi =pr2hx, xi = |r|phx, xi.Triangle inequality (follows from Cauchy-Schwarz’s):kx + yk2= hx + y, x + yi= hx, xi + hx, yi + hy, xi + hy, yi≤ hx, xi + |hx, yi| + |hy, xi| + hy, yi≤ kxk2+ 2kxk kyk + kyk2= (kxk + kyk)2.Examples. • The length of a vector in Rn,|x| =px21+ x22+ · · · + x2n,is the norm induced by the dot productx · y = x1y1+ x2y2+ · · · + xnyn.• The norm kf k2=Zba|f (x)|2dx1/2on thevector space C [a, b] is induced by the inner producthf , gi =Zbaf (x)g(x) dx.AngleSince |hx, yi| ≤ kxk kyk, we can define the anglebetween nonzero vectors in any vector space withan inner product (and induced norm):∠(x, y) = arccoshx, yikxk kyk.Then hx, yi = kxk kyk cos ∠(x, y).In particular, vectors x and y are orthogonal(denoted x ⊥ y) if hx, yi = 0.xx + yyPythagorean Law:x ⊥ y =⇒ kx + yk2= kxk2+ kyk2Proof: kx + yk2= hx + y, x + yi= hx, xi + hx, yi + hy, xi + hy, yi= hx, xi + hy, yi = kxk2+ kyk2.Orthogonal setsLet V be an inner product space with an innerproduct h·, ·i and the induced norm k · k.Definition. A nonempty set S ⊂ V of nonzerovectors is called an orthogonal set if all vectors inS are mutually orthogonal. That is, 0 /∈ S andhx, yi = 0 for any x, y ∈ S, x 6= y.An orthogonal set S ⊂ V is called orthonormal ifkxk = 1 for any x ∈ S.Remark. Vectors v1, v2, . . . , vk∈ V form anorthonormal set if and only ifhvi, vji =1 if i = j0 if i 6= jExamples. • V = Rn, hx, yi = x · y.The standard basis e1= (1, 0, 0, . . . , 0),e2= (0, 1, 0, . . . , 0), . . . , en= (0, 0, 0, . . . , 1).It is an orthonormal set.• V = R3, hx, yi = x · y.v1= (3, 5, 4), v2= (3, −5, 4), v3= (4, 0, −3).v1· v2= 0, v1· v3= 0, v2· v3= 0,v1· v1= 50, v2· v2= 50, v3· v3= 25.Thus the set {v1, v2, v3} is orthogonal but notorthonormal. An orthonormal set is formed bynormalized vectors w1=v1kv1k, w2=v2kv2k,w3=v3kv3k.• V = C [−π, π], hf , g i =Zπ−πf (x)g(x) dx.f1(x) = sin x, f2(x) = sin 2x, . . . , fn(x) = sin nx, . . .hfm, fni =Zπ−πsin(mx) sin(nx) dx =π if m = n0 if m 6= nThus the set {f1, f2, f3, . . . } is orthogonal bu t notorthonormal.It is orthonormal with respect to a scaled innerproducthhf , g ii =1πZπ−πf (x)g(x) dx.Orthogonality =⇒ linear independenceTheorem Suppose v1, v2, . . . , vkare nonzerovectors that form an orthogonal set. Thenv1, v2, . . . , vkare linearly independent.Proof: Suppose t1v1+ t2v2+ · · · + tkvk= 0for some t1, t2, . . . , tk∈ R.Then for any index 1 ≤ i ≤ k we haveht1v1+ t2v2+ · · · + tkvk, vii = h0, …
View Full Document
Unlocking...