Pre-lecture: the shocking state of our ignoranceQ: How fast can we solveN l inear equations in N unknowns?Estimated cost of Gaussian el imination: ∗ ∗∗0 ∗ ∗∗ 0 ∗ ∗∗• to create the zeros below the pivot:on the order of N2operations• if th ere is N pivots:on the order of N · N2= N3op’s• A more careful count places the cost at ∼13N3op’s.• For large N , it is only the N3that matters.It says that ifN → 10N then we have to work 1000 times as hard.That’s not optima l! We can do better than Gaussian elim ination:• Strassen algorithm (1969): Nlo g27= N2 .8 0 7• Coppersmith–Winograd algorithm (1990): N2.3 7 5•St others–Will iams–Le Gall (2014): N2.3 7 3Is N2possible? We have no idea! (b etter is im p ossible; why?)Good news for applications: (w ill see an exam ple soon)• Matrices typically h ave lots of structure and zeroswhich makes solving so much faster.Ar min [email protected]• Help sessions in 441 AH: MW 4-6pm, TR 5-7pmReview• A system such as2x − y = 1x + y = 5can be w ritten in vector form asx21+ y−11=15.• The left-ha nd side is a linear combination of the vectors21and−11.The row and column pi ctureExample 1. We can thi nk of the linear system2x − y = 1x + y = 5in two diffe r e n t geometric ways. Here, there is a unique solut ion: x = 2, y = 3.Row picture.•Each equa tion defines a line in R2.• Which points lie on the intersectionof these lines?• (2, 3) is the (only) intersection ofthe two lines 2x − y = 1 and x +y = 5.1234512345Ar min [email protected] picture.•The system can be written asx21+ y−11=15.• Which linear combinations of21and−11produce15?• (2, 3) are the co e ffi c ients of the(only) such linear combi n ation.-3-2-10123412345Example 2. Consider the vectorsa1=103, a2=4214, a3=3610, b =−18−5.Determine if b is a linear combination of a1, a2, a3.Solution. Vector b is a linear combination of a1, a2, a3if we can find weights x1, x2,x3such that:x1103+ x24214+ x33610=−18−5This vector equation corresponds to t he linear system:x1+4x2+3x3= −1+2x2+6x3= 83x1+14x2+10x3= −5Corresponding augmented matrix:1 4 3 −10 2 6 83 14 10 −5Note that we are looking for a linear combinatio n of the first three columns whichAr min [email protected] c e s the last column.Such a combination existsthe system is c onsistent.Row reduction to echelon form:1 4 3 − 10 2 6 83 14 10 −5 1 4 3 −10 2 6 80 2 1 −2 1 4 3 − 10 2 6 80 0 −5 −10Since this system is co nsistent, b is a linear comb ination of a1, a2, a3.[It is consistent , because there is no row of th e form[0 0 0 b] with b0.]Example 3. In the previous example, express b as a linear combination of a1, a2, a3.Solution. The reduc e d ech e lon form is:1 4 3 −10 2 6 80 0 −5 −10 1 4 3 −10 1 3 40 0 1 2 1 4 0 −70 1 0 −20 0 1 2 1 0 0 10 1 0 −20 0 1 2We rea d off the solution x1= 1, x2= −2, x3= 2, which yields103− 24214+ 23610=−18−5.SummaryA vector equationx1a1+ x2a2++ xmam= bhas the same solution set as the linear syste m with augmen ted matrix| | | |a1a2amb| | | |.In particular, b can be generated by a line ar combina tion of a1, a2,, amif a n d onlyif this linear system is consistent.Ar min [email protected] span of a set of vectorsDefinition 4. The span of vectors v1, v2,, vmis the set of all their linear combina-tions. We denote it by span{v1, v2,, vm}.In other words,span{v1, v2,, vm} is the set of all vectors of the formc1v1+ c2v2++ cmvm,where c1, c2,, cmare scalars.Example 5.(a)Describe spann21ogeometrica lly.The span consists of all vectors of the formα ·21.As points inR2, this is a line.(b) Describe spann21,41ogeometrica lly.The span is all ofR2, a plane.That’s because any vector inR2canbe wr itten a sx121+ x241.-2-11234-1.0-0.50.51.01.52.0Le t’s show this without relying on our geometric intuition: letb1b2an y vector.24 b111 b2 "24 b10−1 b2−12b1#is consistentHence,b1b2is a linear combination of21an d41.(c) Describe spann21,42ogeometrica lly.Note that42= 2 ·21. Hence, the span is as in (a).Again, we can also see this after row reduction: letb1b2an y vector.24 b112 b2 "24 b100 b2−12b1#is not consistent for allb1b2b1b2is in the span of21and42only if b2−12b1= 0 (i.e. b2=12b1).So the span consists of vectors"b112b1#= b1"112#.Ar min [email protected] single (nonzero) vector always spans a line, two vectors v1, v2usually span a planebut it could also be just a line (ifv2= αv1).We will come back to this when we discuss dimension and linear inde pendence.Example 6. Is span(2−11,4−21)a line or a plane?Solution. The span is a plane unless, for some α,4−21= α ·2−11.Looking at th e first entry, α = 2, but that d oes not work for the third entry. Hence,there is no suchα. The span is a plane.Example 7. ConsiderA =1 23 10 5, b =8317.Is b in the plane spanned by the columns of A?Solution. b in the plane span ned by the columns of A if and only if1 2 83 1 30 5 17is consistent.To find out, we row reduce to an eche lon form:1 2 83 1 30 5 17 12 80 −5 −210 5 17 12 80 −5 −210 0 −4From the last row , we see that the s ystem is inconsistent. Hence, b is not in the planespanned by the columns ofA.Ar min [email protected] and summary• The span of vectors a1, a2,, amis t he set o f al l th eir linear combinations.• S ome vector b is in span{a1, a2,, am} if and only if th e r e is a solution to thelinear system with augmented ma t r i x| | | |a1a2amb| | | |.◦ Each solution corresp o n ds to the weights in a linear combination of the a1, a2,,amwhich gives b.◦ This gives a second geometric way to think of linear systems!Ar min
View Full Document
Unlocking...