MATH 304Linear AlgebraLecture 18:Orthogonal complement (continued).Orthogonal projection.Least squares problems.Euclidean structureEuclidean structure in Rnincludes:• length of a vector: |x|,• angle between vectors: θ,• dot product: x · y = |x||y| cos θ.A BCθyxLength and distanceDefinition. The length of a vectorv = (v1, v2, . . . , vn) ∈ Rniskvk =pv21+ v22+ ··· + v2n.The distance between vectors/points x and y isky − xk.Properties of length:kxk ≥ 0, kxk = 0 only if x = 0 (positivity)krxk = |r|kxk (homogeneity)kx + yk ≤ kxk + kyk (triangle inequality)Scalar productDefinition. The scalar product of vectorsx = (x1, x2, . . . , xn) and y = (y1, y2, . . . , yn) isx · y = x1y1+ x2y2+ ··· + xnyn.Properties of scalar product:x · x ≥ 0, x · x = 0 only if x = 0 (positivity)x · y = y ·x (symmetry)(x + y) · z = x · z + y ·z (distributive law)(rx) · y = r (x · y) (homogeneity)In particular, x · y is a bilinear function (i.e., it isboth a linear function of x and a linear function of y).AngleCauchy-Schwarz inequality: |x · y| ≤ kxkkyk.By the Cauchy-Schwarz inequality, for any nonzerovectors x, y ∈ Rnwe havecos θ =x · ykxkkykfor a unique 0 ≤ θ ≤ π.θ is called the angle between the vectors x and y.The vectors x and y are said to be orthogonal(denoted x ⊥ y) if x · y = 0 (i.e., if θ = 90o).OrthogonalityDefinition 1. Vectors x, y ∈ Rnare said to beorthogonal (denoted x ⊥ y) ifx · y = 0.Definition 2. A vector x ∈ Rnis said to beorthogonal to a nonempty set Y ⊂ Rn(denotedx ⊥ Y ) if x ·y = 0 for any y ∈ Y .Definition 3. Nonempty sets X , Y ⊂ Rnare saidto be orthogonal (denoted X ⊥ Y ) if x · y = 0for any x ∈ X and y ∈ Y .Orthogonal complementDefinition. Let S ⊂ Rn. The orthogonalcomplement of S, denoted S⊥, is the set of allvectors x ∈ Rnthat are orthogonal to S.Theorem 1 (i) S⊥is a subspace of Rn.(ii) Span(S)⊥= S⊥.(iii) (S⊥)⊥= Span(S).Theorem 2 If V is a subspace of Rn, then(i) (V⊥)⊥= V ,(ii) V ∩ V⊥= {0}.VV⊥0Fundamental subspacesDefinition. Given an m×n matrix A, letN(A) = {x ∈ Rn| Ax = 0},R(A) = {b ∈ Rm| b = Ax for some x ∈ Rn}.R(A) is the range of a linear mapping L : Rn→ Rm,L(x) = Ax. N(A) is the kernel of L.Also, N(A) is the nullspace of the matrix A whileR(A) is the column space of A. The row space ofA is R(AT).The subspaces N(A), R(AT) ⊂ RnandR(A), N(AT) ⊂ Rmare fundamental subspacesassociated to the matrix A.Theorem N(A) = R(AT)⊥, N(AT) = R(A)⊥.That is, the nullspace of a matrix is the orthogonalcomplement of its row space.Proof: The equality Ax = 0 means that the vector x isorthogonal to rows of the matrix A. Therefore N(A) = S⊥,where S is the set of rows of A. It remains to note thatS⊥= Span(S)⊥= R(AT)⊥.Corollary Let V be a subspace of Rn. Thendim V + dim V⊥= n.Proof: Pick a basis v1, . . . , vkfor V . Let A be the k×nmatrix whose rows are vectors v1, . . . , vk. Then V = R(AT),hence V⊥= N(A). Consequently, dim V and dim V⊥arerank and nullity of A. Therefore dim V + dim V⊥equals thenumber of columns of A, which is n.Orthogonal projectionTheorem 1 Let V be a subspace of Rn. Thenany vector x ∈ Rnis uniquely represented asx = p + o, where p ∈ V and o ∈ V⊥.Idea of the proof: Let v1, . . . , vkbe a basis for V andw1, . . . , wmbe a basis for V⊥. Then v1, . . . , vk, w1, . . . , wmis a basis for Rn.In the above expansion, p is called the orthogonalprojection of the vector x onto the subspace V .Theorem 2 kx − vk > kx − pk for any v 6= p in V .Thus kok = kx − pk = minv∈Vkx − vk is thedistance from the vector x to the subspace V .VV⊥opxOrthogonal projection onto a vectorLet x, y ∈ Rn, with y 6= 0.Then there exists a unique decomposition x = p + osuch that p is parallel to y and o is orthogonal to y.ypxop = orthogonal projection of x ont o yOrthogonal projection onto a vectorLet x, y ∈ Rn, with y 6= 0.Then there exists a unique decomposition x = p + osuch that p is parallel to y and o is orthogonal to y.We have p = αy for some α ∈ R. Then0 = o · y = (x − αy) · y = x ·y − αy · y.=⇒ α =x · yy · y=⇒p =x · yy · yyProblem. Find the distance from the pointx = (3, 1) to the line spanned by y = (2, −1).Consider the decomposition x = p + o, where p is parallel toy while o ⊥ y. The required distance is the length of theorthogonal component o.p =x · yy · yy =55(2, −1) = (2, −1),o = x − p = (3, 1) − (2, −1) = (1, 2), kok =√5.Problem. Find the point on the line y = −x thatis closest to the point (3, 4).The required point is the projection p of v = (3, 4) on thevector w = (1, −1) spanning the line y = −x.p =v · ww · ww =−12(1, −1) =−12,12Problem. Let Π be the plane spanned by vectorsv1= (1, 1, 0) and v2= (0, 1, 1).(i) Find the orthogonal projection of the vectorx = (4, 0, −1) onto the plane Π.(ii) Find the distance from x to Π.We have x = p + o, where p ∈ Π and o ⊥ Π.Then the orthogonal projection of x onto Π is p andthe distance from x to Π is kok.We have p = αv1+ βv2for some α, β ∈ R.Then o = x − p = x − αv1− βv2.o · v1= 0o · v2= 0⇐⇒α(v1· v1) + β(v2· v1) = x · v1α(v1· v2) + β(v2· v2) = x · v2x = (4, 0, −1), v1= (1, 1, 0), v2= (0, 1, 1)α(v1· v1) + β(v2· v1) = x · v1α(v1· v2) + β(v2· v2) = x · v2⇐⇒2α + β = 4α + 2β = −1⇐⇒α = 3β = −2p = 3v1− 2v2= (3, 1, −2)o = x − p = (1, −1, 1)kok =√3Problem. Let Π be the plane spanned by vectorsv1= (1, 1, 0) and v2= (0, 1, 1).(i) Find the orthogonal projection of the vectorx = (4, 0, −1) onto the plane Π.(ii) Find the distance from x to Π.Alternative solution: We have x = p + o, where p ∈ Π ando ⊥ Π . Then the orthogonal projection of x onto Π is p andthe distance from x to Π is kok.Notice that o is the orthogonal projection of x onto theorthogonal complement Π⊥. In the previous lecture, we foundthat Π⊥is the line spanned by the vector y = (1, …
View Full Document
Unlocking...