Math 19b: Linear Algebra with Probability Oliver Knill, Spring 2011Lecture 14: BasisRecall that subset X of Rnwhich is closed under addition and scalar multiplicationis called a linear subspace of Rn. We have to check three conditions: (a) 0 ∈ V ,(b) ~v + ~w ∈ V if ~v, ~w ∈ V . (b) λ~v ∈ V if ~v and λ is a r eal number.1 The image and kernel of a transformation are linear spaces. This is an import ant examplesince this is how we describe linear spaces, either as the image of a linear transformationor t he kernel of a linear transformations. Both are useful and they are somehow dual toeach other. The kernel is associated to row vectors because we are perpendicular to allrow vectors, the image is associated to column vectors because we are perpendicular to allcolumn vectors.A set B of vectors ~v1, . . . , ~vmis called basis of a linear subspace X of Rnif they arelinear independent and if they span the space X. Linear independent means thatthere are no nontrivial linear relations ai~v1+ . . . + am~vm= 0. Spanning the spacemeans that very vector ~v can be written as a linear combination ~v = a1~v1+. . .+am~vmof basis vectors.2 Two nonzero vectors in the plane form a basis if they are not parallel.3 The standard basis vectors e1=10...0, e2=01...0, . . . , en=00...1form a basis in Rn.Given a basis B in V . Every vector in V can be written in a unique manner as alinear combination of vectors in B.To see this, assume ~v is written in two different ways~v = a1v1+ a2v2+ ... + anvn= b1v1+ b2v2+ ... + bnvn.a Then (a1− b1)v1+ (a2− b2)v2+ ...(an− bn)vn= 0. But this shows that the vectors are notlinearly independent.4 Given a proba bility space with 4 elements. Any random variable can be written in a uniqueway as a linear combination of t he 4 random variables1−100,1100,001−1,0011.In general, one a finite probability space. A basis defines n random variables such that everyrandom variable X can be written as a linear combination of X1, ..., Xn.A set of random variables X1, .., Xnwhich form a basis on a finite probability spaceΩ = {1, ..., n } describe everything. Every random variable X which we want tocompute can be expressed using these random variables.Given a bunch of vectors in a linear space V , we can construct a basis of V bysticking the vectors as columns into a matrix A, then pick the pivot columns of A.5 Find a basis of the space V spanned by the three vectorsA =123−2,369−6,0510.Solution: For m the matrixA =1 3 02 6 53 9 1−2 −6 0.Row reduction shows that the first and third vector span the space V . This is a basis forV .A n × n matrix A =| | |~v1~v2. . . ~vn| | |is invertible if and only if ~v1, . . . , ~vnis abasis in Rn.6 Find a basis for the image and the kernel of A =0 0 11 1 01 1 1. Solution: In reduced rowechelon form is B = rref(A) =1 1 00 0 10 0 0. To determine a basis of the kernel we writeBx = 0 as a system of linear equations: x + y = 0, z = 0. The variable y is the freevariable. With y = t, x = −t is fixed. The linear system rref(A)x = 0 is solved by~x =xyz= t−110. So, ~v =−110is a basis of the kernel. Because the first and thirdvectors in rref(A) are pivot columns, the vectors ~v1=011, ~v2=101form a basis of theimage of A.Why do we not just always stick to the standard basis vectors ~e1, . . . , ~en? The reason forthe need of mor e general basis vectors is tha t they allow a more flexible adaptation tothe situation. In geometry, the reflection of a ray at a plane or at a curve is better donein a basis adapted to the situation. For differential equations, the system can be solved ina suitable basis. Basis also matters in statistics. Given a set of r andom variables, we oftencan find a basis for them which consists of uncorrelated vectors.How do we check that a set of vectors form a basis in Rn?A set of n vectors ~v1, ..., ~vnin Rnform a basis in Rnif and only if the matrix Acontaining the vectors as column vectors is invertible.7 The vectors ~v1=011, ~v2=101and ~v2=121form a basis of R3because the matrixA =0 1 11 0 21 1 1is invertible.More generally, the pivot columns of an arbitrary matrix A for m a basis for the image of A.Since we represent linear spaces always as the kernel or image of a linear map, the pro blemof finding a basis to a linear space is always the problem o f finding a basis for the image orfinding a basis fo r the kernel of a matrix.Homework due March 2, 20111 Find a basis for the image and kernel of the Chess matrix:A =1 0 1 0 1 0 1 00 1 0 1 0 1 0 11 0 1 0 1 0 1 00 1 0 1 0 1 0 11 0 1 0 1 0 1 00 1 0 1 0 1 0 11 0 1 0 1 0 1 00 1 0 1 0 1 0 1and verify the rank-nullety theorem in this case.2 Find a basis for the set of vectors perpendicular to the image of A, where A is thePascal matrix.A =0 0 0 0 1 0 0 0 00 0 0 1 0 1 0 0 00 0 1 0 2 0 1 0 00 1 0 3 0 3 0 1 01 0 4 0 6 0 4 0 1.3 Verify that a vector is in the kernel of a mat r ix ATif and only if it perpendicular tothe image of A.Verify it in the following concrete exampleA =1 3 22 6 03 9 14 1 1.The matrix ATdenotes the matrix where the columns and rows of A are switched. Itis called the transpose of the matrix