Lecture #4:0.0.1 Divide & Conquer Method:We now take up the concept of divide and conquer. We giv e sev eral examples inwhat follows. The basic idea is to divide up the problem into smaller problemsand recursively solve each and combine the solutions to make up the solutionof the larger problem. Some times, this reduces the computational effort and itis these cases that we are interested in. The first few examples from the firstlecture that you have most likely seen before this class were: (i)Binary Search;(ii) Merge Sort; and (iii) The Tiling Problem from lecture #1. Now we do afew more.Example 1 Given an array of numb ers A[1, 2, ..., n], find the maximum andthe minimum of the set of the given numbers using only pairwise co mparisons.We assume that n is a power of 2.Algorithm 2 (A) Starting with both Max and Min set to A[1],wecompareeach element in the array to the current values of these and if A[i] >Max,change Max to A[i];ifA[i] <Min, change Min to A[i]. We need to do thesecond comparison only if the first does not change the value of Max.Algorithm 3 (A’) First find Max and this takes (n − 1) comparisons. Nowfind Min among the remaining elements and this takes (n − 2) comparisonsfor a total of 2n − 3 comaprisons.What is the (worst case) number of comparisons in Algorithm A? . It shouldbe clear that any algorithm for finding the Max is Ω(n).Thus,theabovealgorithms are ”optimal” as regards the ”order”. But may be there is anotheralgorithm that is also O(n) but the constant is lower. This is indeed the caseas we shall see below.Algorithm 4 (B) Consider the relation:Max(A[1, 2, ...n]) = max{Max(A[1, 2, ...,jn2k]),Max(A[jn2k+1, ..., n])}Min(A[1, 2, ...n]) = min{Min(A[1, 2, ...,jn2k]),Min(A[jn2k+1,...,n])}So if we know the results for the two half arrays, by doing two more com-parisons, we get the results for the full array. This gives the following recursion:t(n)=0 if n =11 if n =2t(¥n2¦)+t(§n2¨)+2if n ≥ 31We use the iteration method to solve this recursion (Why not use the MasterTheorem?).Sincewehaveassumedthatn is a power of 2,letn =2kfor somek. Then,t(n)=2+2t(n2)= 2+2[2+2t(n4)]= 2+4+8+16+..=2k−1t(2) +k−1Xi=12ik−1Xi=12i= 2+4+8+... +2k−1=2[1+2+4+... +2k−2]=2(2k−1− 1) = 2k− 2Therefore, t(n)=2k+2k−1− 2=n +n2− 2=32n − 2 < 2n − 3.Exercise 5 Show by substitution method that for general values of n, t(n)=§32n¨− 2.Algorithm B does less work than Algorithms A’. The savings are only about25% and this may be offset by the difficulty in implementing algorithm B. Thereis a much simpler algorithm that achieves the same bound as algorithm B.Algorithm 6 (C) Define two arra ys B[1, 2, ...,n2] and C[1, 2, ...,n2] of half thelength of the array A as follows: B[i]=max(A[2i − 1],A[2i]); C[i]=min(A[2i −1],A[2i]); 1 ≤ i ≤n2. Find the maximum of the numbers in the array B and theminimum of the numbers in the array C. The total effort here is also§32n − 2¨,but the algorithms is much simpler!But we are really looking for much bigger savings! Let’s move on to otherexamples.Example 7 Fast Integer MultiplicationLet X = x1x2x3...xnand Y = y1y2y3...ynbe two n-digit numbers that wewish to multiply. The ordinary method of computing the product X ·Y involvesn2multiplication of single digit numbers yi(1 ≤ i ≤ n) with xj(1 ≤ j ≤ n).Letus consider divide and conquer to develop a faster algorithm. Divide both Xand Y into two numbers with half the number of digits in the original numbers.So we get:X1= x1x2x3...xn2X2= xn2+1,xn2+2, ...xnY1= y1y2y3...yn2Y2= yn2+1yn2+2...yn2We assume that n is a power of t wo for the sake of convenience. Thus,X = X110n2+ X2Y = Y110n2+ Y2andX · Y = X1· Y110n+[X1· Y2+ X2· Y1]10n2+ X2· Y2The last expression involves four products of in tegers of half the size com-pared to the original problem. The number of steps for adding these termstogether and for doing the left shifting to get X1· Y (1)10nfrom X1· Y (1) is cnfor some positive c. Hence, we obtain the following recurrence:t(n)=4t(n2)+cnIf we use the master theorem to solve this recurrence, a =4,b =2;logba =2; f(n)=Θ(n) and so we are in Case 1. Hence t(n)=Θ(n2).Thisisnoimprovement! If you look carefully, we need only to compute three terms:X1·Y1,[X1· Y2+ X2· Y1],andX2· Y2. How we do it is up to us!. Notice[X1· Y2+ X2· Y1]=[X1+ X2] · [Y1+ Y2] − X1· Y1− X2· Y2Hence, if we compute the three products X1· Y1, [X1+ X2] · [Y1+ Y2] andX2· Y2and two additions (subtractions actually) above, we get all the requiredterms in X · Y . With this, our recurrence becomest(n)=3t(n2)+cnNow we apply the master theorem, (again Case 1) we get t(n)=Θ(nlg 3)which is better than before. We hav e indeed conquered! In order to see whetherw e can do even better (for example can we split the numbers into three equalparts and do the same thing) , we need to understand why this worked. Forthis purpose, letX(t)=X1· t + X2Y (t)=Y1· t + Y2Z(t)=X(t) · Y (t)= X1· Y1t2+[X1· Y2+ X2· Y1]t + X2· Y2= at2+ bt + cThus, Z(t) is a second degree polynomial in t and has three coefficients a, b,and c.IfweknowthevaluesofZ(t) for three different values of t then we cancalculate the coefficients a, b, and c.Thethreevaluesoft are normally chosenas −1, 0, and 1. We use La Grange’s formula:3Z(t)=1Xi=−11Yj6=i;j=−1·(t − j)(i − j)Z(i)¸= {t−1·t − 1−1 − 1Z(−1)} + {t +11·t − 1−1Z(0)} + {t +12·t1Z(1)}=t2− t2Z(−1) + (−t2+1)Z(0) +t2+ t2Z(1)=[12Z(−1) − Z(0) +12Z(1)]t2+[−12Z(−1) +12Z(1)]t + Z(0)= at2+ bt + cThe three values Z(−1),Z(0), and Z(1) are computed b y three multiplica-tions on integers half the size of the original numbers.Z(−1) = X(−1) · Y (−1)Z(0) = X(0) · Y (0)Z(1) = X(1) · Y (1)Note thatZ(−1) = X(−1) · Y (−1)=[−X1+ X2] · [−Y1+ Y2]= X1· Y1− [X1· Y2+ X2· Y1]+X2· Y2Z(0) = X(0) · Y (0)= X2· Y2Z(1) = X(1) · Y (1)=[X1+ X2] · [Y1+ Y2]If we let t =10n2,wegetZ = X · Y = a10n+ b10n2+ c.You can do the same process by dividing each num ber into three parts. theanalysis is similar. In general, doing this we get t(n)=Θ(nlogk(2k−1)) and wecan make this Θ(n1+) for arbitrary positive by letting k become large. Thiscompletes this example. What we have indirectly shown is how to multiply twopolynomial functions.In the above discussion as applied to multiplying two integers of equal size,we have been some what sloppy. X(1) + X(2) may have more digits than eachof these numbers. What if the size
View Full Document
Unlocking...