Math 19b: Linear Algebra with Probability Oliver Knill, Spring 2011Lecture 15: DimensionRemember that X ⊂ Rnis called a linear space if~0 ∈ X and if X is closed under addition andscalar multiplication. Examples are Rn, X = ker(A), X = im(A), or the row space of a matrix.In order to describe linear spaces, we had the notion of a basis:B = {~v1, . . . ,~vn} ⊂ X is a basis if two conditions are satisfied: B is linear independent meaningthat c1~v1+ ... + cn~vn= 0 implies c1= . . . = cn= 0. Then B span X: ~v ∈ X then ~v =a1~v1+ . . . + an~vn. The spanning condition for a basis assures that there are enough vectorsto represent any other vector, the linear independence condition assures that there are not toomany vectors. Every ~v ∈ X can be written uniquely as a sum ~v = a1~v1+ . . . + an~vnof basisvectors.The dimension of a linear space V is the number of basis vectors in V .The dimension o f three dimensional space is 3. The dimension is independent on where the spaceis embedded in. For example: a line in the plane and a line embedded in space have both thedimension 1.1 The dimension of Rnis n. The standard basis is10...0,01...0, ···,00...1.2 The dimension of {0 } is zero. The dimension of any line 1. The dimension of a plane is 2.3 The dimension of the image of a matrix is the number of pivot columns. We can constructa basis of the kernel and imag e of a linear transformation T (x) = Ax by forming B = rr efA.The set of Pivot columns in A form a basis of the image of T .4 The dimension of the kernel of a matrix is the number of free variables. It is also callednullety. A basis for the kernel is obtained by solving Bx = 0 and introducing free variablesfor the redundant columns.Given a basis A = {v1, ..., vn} and a basis B = {w1, ..., .wm} of X, then m = n.Lemma: if q vectors ~w1, ..., ~wqspan X and ~v1, ..., ~vpare linearly independent in X, then q ≥ p.Assume q < p. Because ~wispan, each vector ~vican be written asPqj=1aij~wj= ~vi. Now do Gauss-Jordan elimination of the augmented (p ×(q + n))-matrix to this system:a11. . . a1q| ~vT1. . . . . . . . . | . . .ap1. . . apq| ~vTp,where ~vTiis the vector ~viwritten as a row vector. Each row of A of thishA|bicontains somenonzero entry. We end up with a matrix, which contains a last row0 ... 0 | b1~wT1+ ... + bq~wTqshowing that b1~wT1+ ···+ bq~wTq= 0. Not all bjare zero because we had to eliminate some nonzeroentries in the last row of A. This nontrivial relation of ~wTi(and the same relation for columnvectors ~w) is a contradiction to the linear independence of the ~wj. The assumption q < p can notbe true.To prove the proposition, use the lemma in two ways. Because A spans and B is linearly inde-pendent, we have m ≤ n. Because B spans and A is linearly independent, we have n ≤ m.The following theorem is also called the rank-nullety theorem because dim(im(A)) is the rankand dim(ker(A))dim(ker(A)) is the nullety.Fundamental theorem of linear algebra: Let A : Rm→ Rnbe a linear map.dim(ker(A)) + dim(im(A)) = mThere are n columns. dim(ker(A)) is the number of columns without leading 1, dim(im(A)) is thenumber of columns with leading 1.5 If A is an invertible n × n matrix, then the dimension of the imag e is n and that thedim(ker)(A) = 0.6 The first grade multiplication table matrixA =1 2 3 4 5 6 7 8 92 4 6 8 10 12 14 16 183 6 9 12 15 18 21 24 274 8 12 16 20 24 28 32 365 10 15 20 25 30 35 40 456 12 18 24 30 36 42 48 547 14 21 28 35 42 49 56 638 16 24 32 40 48 56 64 729 18 27 36 45 54 63 72 81.has rank 1. The nullety is therefore 8.7 Are there a 4 × 4 matrices A, B of ranks 3 and 1 such that ran(AB) = 0? Solution. Yes,we can even find examples which are diagonal.8 Is there 4 × 4 matrices A, B of rank 3 and 1 such that ran(AB) = 2? Solution. No, thekernel of B is three dimensional by the rank-nullety theorem. But this also means the kernelof AB is three dimensional (the same vectors a re annihilated). But this implies that therank of AB can maximally be 1.The rank of AB can not be larger than the rank of A or the rank of B.The nullety of AB can no t be smaller than the nullety of B.We end this lecture with an informal remark about fractal dimension:Mathematicians study objects with non-integer dimension since the early 20’th century. The topicbecame fashion in the 1980’ies, when mathematicians star t ed to generate fractals on computers.To define fractals, the notion of dimension is extended. Here is an informal definition which canbe remembered easily and allows to compute the dimension of most ”star fractals” you find onthe internet when searching for fractals. It assumes that X is a bounded set. You can pick upthis definition also in the Startreck movie (2009) when little Spock gets some math and ethicslectures in school. It is the simplest definition and also called box counting dimension in the mathliterature on earth.Assume we can cover X with n = n(r) cubes of size r and not less. The fractaldimension is defined as the limitdim(X) =−log(n)log(r)as r → 0.For linear spaces X, the fractal dimension of X intersected with the unit cube agreeswith the usual dimension in linear algebra.Pro of. Take a basis B = {v1, . . . , vm} in X. We can assume that this basis vectors ar e allorthogonal and each vector has length 1. For given r > 0, place cubes at the lattice pointsPmj=1kjrvjwith integer kj. This covers the intersection X with the unit cube with (C/rm) cubeswhere 1/√m ≤ C ≤√m. The dimension o f X isdim(X) = log(C/rm)/ log(r) = log(C)/ log(r) + mwhich converges to m as r → 0.9 We cover the unit interval [0, 1] with n = 1/r intervals of length r. Now,dim(X) =−log(1/r)log(r)= 1 .10 We cover the unit square with n = 1/r2squares of length r. Now,dim(X) =−log(1/r2)log(r)= 2 .11 The Cantor set is obtained recursively by dividing intervals into 3 pieces and throwingaway t he middle one. We can cover the Cantor set with n = 2kintervals of length r = 1/3kso thatdim(X) =−log(2k)log(1/3k)= log(2)/ log(3) .12 The Shirpinski carpet is constructed recursively by dividing a square in 9 equal squaresand throwing away the middle one, repeating this procedure with each of the squares etc.At the k’th step, we need n = 8ksquares of length r = 1/3kto cover X. The