MATH 311Topics in Applied MathematicsLecture 17:Orthogonal complement.Orthogonal projection.Scalar product in RnDefinition. The scalar product of vectorsx = (x1, x2, . . . , xn) and y = (y1, y2, . . . , yn) isx · y = x1y1+ x2y2+ ··· + xnyn.Properties of scalar product:x · x ≥ 0, x · x = 0 only if x = 0 (positivity)x · y = y · x (symmetry)(x + y) · z = x · z + y · z (distributive law)(rx) · y = r (x · y) (homogeneity)In particular, x · y is a bilinear function (i.e., it isboth a linear function of x and a linear function of y).OrthogonalityDefinition 1. Vectors x, y ∈ Rnare said to beorthogonal (denoted x ⊥ y) ifx · y = 0.Definition 2. A vector x ∈ Rnis said to beorthogonal to a nonempty set Y ⊂ Rn(denotedx ⊥ Y ) if x · y = 0 for any y ∈ Y .Definition 3. Nonempty sets X , Y ⊂ Rnare saidto be orthogonal (denoted X ⊥ Y ) if x · y = 0for any x ∈ X and y ∈ Y .Examples in R3. • The line x = y = 0 isorthogonal to the line y = z = 0.Indeed, if v = (0, 0, z) and w = (x, 0, 0) then v ·w = 0.• The line x = y = 0 is orthogonal to the planez = 0.Indeed, if v = (0, 0, z) and w = (x, y , 0) then v · w = 0.• The line x = y = 0 is not orthogonal to theplane z = 1.The vector v = (0, 0, 1) belongs to both the line and theplane, and v · v = 1 6= 0.• The plane z = 0 is not orthogonal to the planey = 0.The vector v = (1, 0, 0) belongs to both planes andv · v = 1 6= 0.Proposition 1 If X , Y ∈ Rnare orthogonal setsthen either they are disjoint or X ∩Y = {0}.Proof: v ∈ X ∩ Y =⇒ v ⊥ v =⇒ v ·v = 0 =⇒ v = 0.Proposition 2 Let V be a subspace of Rnand Sbe a spanning set for V . Then for any x ∈ Rnx ⊥ S =⇒ x ⊥ V .Proof: Any v ∈ V is represented as v = a1v1+ ··· + akvk,where vi∈ S and ai∈ R. If x ⊥ S thenx · v = a1(x · v1) + ··· + ak(x · vk) = 0 =⇒ x ⊥ v.Example. The vector v = (1, 1, 1) is orthogonal tothe plane spanned by vectors w1= (2, −3, 1) andw2= (0, 1, −1) (because v · w1= v · w2= 0).Orthogonal complementDefinition. Let S ⊂ Rn. The orthogonalcomplement of S, denoted S⊥, is the set of allvectors x ∈ Rnthat are orthogonal to S. That is,S⊥is the largest subset of Rnorthogonal to S.Theorem 1 S⊥is a subspace of Rn.Note that S ⊂ (S⊥)⊥, hence Span(S) ⊂ (S⊥)⊥.Theorem 2 (S⊥)⊥= Span(S). In particular, forany subspace V we have (V⊥)⊥= V .Example. Consider a line L = {(x, 0, 0) | x ∈ R}and a plane Π = {(0, y, z) | y, z ∈ R} in R3.Then L⊥= Π and Π⊥= L.Fundamental subspacesDefinition. Given an m×n matrix A, letN(A) = {x ∈ Rn| Ax = 0},R(A) = {b ∈ Rm| b = Ax for some x ∈ Rn}.R(A) is the range of a linear mapping L : Rn→ Rm,L(x) = Ax. N(A) is the kernel of L.Also, N(A) is the nullspace of the matrix A whileR(A) is the column space of A. The row space ofA is R(AT).The subspaces N(A), R(AT) ⊂ RnandR(A), N(AT) ⊂ Rmare fundamental subspacesassociated to the matrix A.Theorem N(A) = R(AT)⊥, N(AT) = R(A)⊥.That is, the nullspace of a matrix is the orthogonalcomplement of its row space.Proof: The equality Ax = 0 means that the vector x isorthogonal to rows of the matrix A. Therefore N(A) = S⊥,where S is the set of rows of A. It remains to note thatS⊥= Span(S)⊥= R(AT)⊥.Corollary Let V be a subspace of Rn. Thendim V + dim V⊥= n.Proof: Pick a basis v1, . . . , vkfor V . Let A b e the k×nmatrix whose rows are vectors v1, . . . , vk. Then V = R(AT)and V⊥= N(A). Consequently, dim V and dim V⊥are rankand nullity of A. Therefore dim V + dim V⊥equals thenumber of columns of A, which is n.Orthogonal projectionTheorem 1 Let V be a subspace of Rn. Thenany vector x ∈ Rnis uniquely represented asx = p + o, where p ∈ V and o ∈ V⊥.In the above expansion, p is called the orthogonalprojection of the vector x onto the subspace V .Theorem 2 kx − vk > kx − pk for any v 6= p in V .Thus kok = kx − pk = minv∈Vkx − vk is thedistance from the vector x to the subspace V .Orthogonal projection onto a vectorLet x, y ∈ Rn, with y 6= 0.Then there exists a unique de composition x = p + osuch that p is parallel to y and o is orthogonal to y.ypxop = orthogonal projection of x onto yOrthogonal projection onto a vectorLet x, y ∈ Rn, with y 6= 0.Then there exists a unique de composition x = p + osuch that p is parallel to y and o is orthogonal to y.We have p = αy for some α ∈ R. Then0 = o · y = (x − αy) · y = x · y − αy · y.=⇒ α =x · yy · y=⇒p =x · yy · yyProblem. Find the distance from the pointx = (3, 1) to the line spanned by y = (2, −1).Consider the decomposition x = p + o, where p is parallel toy while o ⊥ y. The required distance is the length of theorthogonal component o.p =x · yy · yy =55(2, −1) = (2, −1),o = x − p = (3, 1) − (2, −1) = (1, 2), kok =√5.Problem. Find the point on the line y = −x thatis closest to the point (3, 4).The required point is the projection p of v = (3, 4) on thevector w = (1, −1) spanning the line y = −x.p =v · ww · ww =−12(1, −1) =−12,12Problem. Let Π be the plane spanned by vectorsv1= (1, 1, 0) and v2= (0, 1, 1).(i) Find the orthogonal projection of the vectorx = (4, 0, −1) onto the plane Π.(ii) Find the distance from x to Π.We have x = p + o, where p ∈ Π and o ⊥ Π.Then th e orthogonal projection of x onto Π is p andthe distance from x to Π is kok.We have p = αv1+ βv2for some α, β ∈ R.Then o = x − p = x − αv1− βv2.o · v1= 0o · v2= 0⇐⇒α(v1· v1) + β(v2· v1) = x · v1α(v1· v2) + β(v2· v2) = x · v2x = (4, 0, −1), v1= (1, 1, 0), v2= (0, 1, 1)α(v1· v1) + β(v2· v1) = …