1Linear AlgebraChen YuIndiana UniversityVectors  nibacbacnibacbaciiiiii,...,1,,...,1,=−=⇔−==+=⇔+= niababii,...,1, ==⇔=εε   Linear Combinations +++= + mmmmvuvuvuvvvuuuβαβαβαβα............22112121Vector Inner ProductuvvuvuvuyxyxTTniiiTniii====⋅===11σσ2The L2 NormxxxxxxxxxxTniin=⋅==+++==212/122/1222212)()...(Matrix• Addition and subtraction• Multiplication by a Scalar• Matrix TransposeMatrix: column vectors =  ×=×× =  == ==mnmnmmnnnnnjjijinbbbxxxaaaaaaaaamnnmbxxxaaaxabAxbaaaA........................]1[]1[][......;...212121222211121121)()2()1(1)()2()1(Linear Independence• Two vectors are not independent if they lie along the same line.  == =2222;111xyx3Matrix Multiplication• Multiplying a matrix by a vector is a special case of matrix multiplication where y = Ax• This can be written as:NNixaxaxay +++= ...2211Alternatively we can see the transformation as linear combination of the columns of A===NkjkjiMixay1,,...,1Coordinate Systems• The vectors ai have a special interpretation as a coordinate system or basis for a multidimensional space. For example, in the traditional basis in three dimensions,  = = =100,010,001321aaa• This basis is orthogonal, since for all i and j such that ji≠0=⋅jiaa332211yayayay++=• The basis vector allow y to be written as Other Bases• A non-orthogonal basis would also work. would still allow y to be represented (although the coefficients would of course be different). However the matrixwould not work because there is no way of representing the thirdcomponent.  −−=100110101A −=000110101ALinear Independence• Consider two linearly independent vectors, u and v, if a third vector, w, cannot be expressed as a linear combination of u and v, then the set {u,v,w} is linearly independent.4Linear Independence• To represent n-dimensional vectors, the basis must span the space. A general condition for this is that the columns of A must be linearly independent. Formally this means that the only way you could writewould be the case that 0...)()2(2)1(1=+++nnxaxaxa0...21====naaaLinear Independence =  000...321)()2()1(aaaxxxn• The number of linearly independent vectors is called the rank of the matrix.• When the rank r is less than the dimension N, the vectors are said to span an r-dimensional subspace. Change of Bases• Change of basis, or coordinate transform: If we have data that is defined relative to some basis, we are free to re-map that data into a new basis• Here A defines our new basis. We can always convert back to the original basis via: Axx =**1xAx−=Eigenvectors• For any matrix W there are special vectors v such that: v is rescaled by a constant . The direction of v is not changed. • The vectors v are known as eigenvectors, and the associated scalars are known as eigenvalues. λvWvλ=λ5Example= 114112213Finding Eigenvectors• The linear equation Ax = 0 only has a solution (non-trivial) if the columns of A are linearly dependent. • The columns of A are linearly dependent iff the determinant of A is equal to zero, |A|=0.• Reminder: the determinant of a 2 x 2 matrixis given by |A| = ad - bc =dcbaAFinding EigenvectorsFor a two-dimensional case: OrFor this equation to have a solution, the columns of the matrix must be linearly dependent, and thus |W|=0. Thus, = 21212213vvvvλ= −−00221321vvλλ02)2)(3(=−−−λλFinding Eigenvectors• Substituting into the equation results in• Only one useful equation in two unknowns. Pick V1=1. Then V2=1. Thus the eigenvector associated with is41=λ41=λ11=