CS/Math 240: Introduction to Discrete Mathematics 3/1/2011Lecture 12 : RecurrencesInstructor: Dieter van Melkebeek Scribe: Dalibor Zelen´yDRAFTLast few classes we talked about program correctness. We discussed correctness of non-recursiveas well as of recursive algorithms. In both cases, the proofs of correctness went by induction.As part of a proof of correct ne ss , we show that the algorithm terminates. For example, we provethat a certain lo op has to terminate after some number of iterations, or that the longest chain ofrecursive calls is finite.In practice, we usually want to know more than that an algorithm terminates. We want itto terminate i n a reasonable amount of time. Thus, we want to prove termination in a strongersense. We not only want to know that the algorithm terminates, but we also want to quantify howfast it terminates. This brings us to the question of program e ffici e nc y. In today ’ s lecture we userecurrences to argue efficiency of programs.12.1 Efficiency of AlgorithmsWhen we analyze efficiency of an algorithm, we are concerned with the number of operations of acertain kind performed by that algorithm, and how this number grows as a function of the inputsize. We call this function the complexity of the algorithm.We usually count the number of some elementary operations, that is, the simplest operationsperformed by the algorithm. Examples of such operations are additions of two integers, bit shifts,recursive calls, or comparisons of two values. The elementary operation we choose depends onthe situation. We just need to make sure that we choose an elementary operation that tells ussomething meaningful about the behavior of th e algorithm. For example , proving that solving amultiplication problem requires one multiplication does not tel l us anything useful . On the otherhand, showing t hat it takes n bit shifts to mult i pl y two numbers whose binary representationsconsist of n bits t e l l s us something. Today’s examples use di ffe r ent kinds of el e me ntary operationsthat are commonly used.We mentioned that we were interested in expressing the number of operations in terms of someinput size, n. This value n could be simply the value of an integer input, but also the length of anarray, or t he length of a binary de sc r i pt i on of an integer.12.2 RecurrencesA recurrence is an inductive definition of a sequence. It specifies the first few terms explicitly, andthen expresses the n-th t e r m in terms of earlier te r ms in the sequence.We’ve already seen an e x ampl e , the Fibonacci numbers. We specify explicitly that the first twoterms, F1and F2, are both 1. For future terms, that is, for n ≥ 3, we define Fn= Fn−1+ Fn−2.1Lecture 12: Recurrences 12.2. Recurrences12.2.1 Towers of HanoiRecall the rules and the overall strategy for the towers of Hanoi game.We have three pegs labeled A, B and C, and n disks of increasing size which are stacked onpeg A with the largest disk on the bottom and smallest disk on top. The goal is to bring al l disksfrom peg A to peg C. In each step, we can move one disk from one pe g to another peg, but we cannever move a larger disk on top of a smaller disk.We discussed a recursive solution last time. We first move the top n − 1 disks from peg A topeg B. Afterwards, peg A has only the largest disk on it, and peg C is empty, so we move thelargest disk from peg A to peg C. To complete the solution, we move the n − 1 disks from peg Bto peg C.Last class we described a re c ur si ve algorithm that returns a sequence of moves that solves theproblem with n disks. We repeat this algorithm below for completeness.Algorithm TOWERS(n, A, B, C)Input: n ∈ N, pegs A, B and C, where n disks are on peg AOutput: Sequence of moves that brings the disks from peg A to peg C, using B as anintermediate peg.(1) if n = 0 then return(2) else return TOWERS(n-1,A,C,B )Move top disk of A to CTOWERS(n-1,B,A,C )We now analyze the compl e x i ty of TOWERS in ter m s of the number of disks, n. Recall that weanalyze the complexity in terms of the number of times the algorithm performs some elementar yoperation. Some options for picking an elementary operation are the numbe r of recursive callsmade or the number of moves (i.e., times we say “Move top disk of x to y”) necessary to solve theproblem with n disks. We pick the latter option for our analy s i s.Let M(n) be the number of moves needed to move n disks from peg A to peg C. Let’s find arecurrence that describes M(n).We see M(0) = 0 since there are n o disks to move.We also have M(1) = 1 since we just move the only disk from A to C right away. Since we arelooking for a recurre nc e relation that descri bes M (n), let’s find M (1) by tracing through the callto TOWERS with n = 1 and see if that leads to a recurrence relation for M(n). When TOWERS getscalled with n = 1, we get in the else clause (line 2). There, we make a call to TOWERS with n = 0,which returns ri ght away and produces no moves, then we make one move, and make another callto TOWERS with n = 0, which produces no additi on al moves. Thus, the total number of moves isindeed 1The trace through the recur si ve calls on input n = 1 shows a pattern. There are three parts toa recursive solution to the problem on n moves.1. Move the top n − 1 disks to peg B. This takes M(n − 1) moves.2. Spend one move on moving the largest disk to peg C.3. Move the n − 1 disks from peg B to peg C. This uses M (n − 1) moves.2Lecture 12: Recurrences 12.2. RecurrencesThus, M(n) = M(n − 1) + 1 + M(n − 1) = 2M(n − 1) + 1, and we define M(n) as follows.M(0) = 0 (12.1)M(n) = 2M(n − 1) + 1, n ≥ 1 (12.2)This expression is nice, but we would li ke to know how M(n) behaves as a function of n. Inother words, we want an expr e ssi on for M(n) that only uses n, and does not use M() anywhere.Intuitively, we see that the number of steps roughly doubles as n increases by 1, so we wouldexpect an expression that contains 2nsomewhere in it (or perhaps 2n−1, 22n, or something elsewith 2 in the exponent). To make this intuition clearer, consider, instead, the sequenceM′(1) = 1M′(n) = 2M′(n − 1), n ≥ 2After writing down the first few terms, we conjecture that M′(n) = 2n−1. Thi s is indeed the case,and you can prove it by induction on n. We omit the p r oof.Now let’s get back to finding an expression for M(n). Let’s start by writing down a few values.Since we suspect there i s some dependence on power s of …