IE172 – Algorithms in Systems EngineeringLecture 8Pietro BelottiDept. of Industrial and Systems EngineeringLehigh University3 February 2010Binary Search Trees: next largest/smallestHow do we find the next l argest key value?Basic Idea◮If node x has a right child r(x), then returnMINIMUM(r(x)).◮Otherwise, start walking up until you make the first“right” move.Note that “get next smallest” behaves similarly.Insert and Delete◮Inserting an element is fairly straightforward: Walk to thebottom, and put it at.◮Deleting an element is harder, but made easier by thefollowing fact:Binary Search Tree FactIf a node has two children, then its successor (next largest)has no left child. Its predecessor (next smallest) has no rightchild◮So to delete, splice a node x with ket k(x), splice out itssuccess and put it in node x′s place.red-black Trees◮red-black trees are simply a way to keep binary searchtrees short. (Or balanced)◮Balanced here means that no path on the tree is more thantwice as long as another path.◮An implication of this is that its maximum height is2 lg(n + 1)◮SEARCH(), MINIMUM(), MAXIMUM(), all take O(lg n)◮It’s implementation is complicated, so we won’t cover it◮INSERT(): also runs in O lg(n)◮DELETE(): runs in O lg(n) (but it is more complicated tomaintain the “red-black” property)QuicksortThe quicksort is another sorting algorithm, that als o can beimplemented as a recursive procedure.◮Choose a partition element x of A.◮Partition the input array around x to obtain two subarrays:◮one with all elements Ai≤ x◮one with all elements Ai≥ x◮Recursively call quick sort on the two subarrays.QUICKSORT (A, ℓ, r):if l < r thenq ← PARTITION (A, ℓ, r)QUICKSORT (A, ℓ, q − 1)QUICKSORT (A, q − 1, r)end ifPartitioningOne big advantage of quicksort is that the partitioning (andhence the entire algorithm) can be performedin place.int PARTITION (A, ℓ, r):x ← Ari ← ℓ − 1for j : ℓ ≤ j ≤ r − 1 doif Aj≤ x theni ← i + 1swap Ai↔ Ajend ifend forswap Ai+1↔ Arreturn i + 1Analyzing Quicksort◮How do we prove the correctness of quick sort?◮Does quicksort always terminate?◮Can we do some simple optimization to improveperformance?◮What are the best case, worst case, and average caserunning times?◮How does quicksort perform on special lists, such as thosethat are almost sorted?Importance of the Par titioning Element◮The performance of the algorithm depends primarily onthe chosen partition element.Q.: What is the “best” partition element to select?A.: One dividing the array in two subarr ays of equal size n/2Q.: What is the running time if we always select the “best”?A.: T(n) = 2T(n/2) + Θ(n)Q.: What is the “worst” partition element to select?A.: One dividing the array in two subarr ays of unequal size,say 1 and n − 1Q.: What is the running time in the worst case?A.: T(n) = T(n − 1) + Θ(n) + Θ(1)Q.: What is the running time in the average case?Choosing the Partitioning Elem ent◮We would like the partition element to be as close to themiddle of the array as possible.◮However, we have no way to ensure this in general.◮If the array is randomly ordered, any element will do, sochoose thelast element◮If the array is almost sorted, this will be disastrous!◮To even the playing field, we can simply choose thepartition elementrandomly.More Simple O ptimizati on◮Note that the check if (j == l) in the partitionfunction can be a significant portion of the running time.◮This check is only there in case the partition element is thesmallest element in the array.◮An approach is to ensure that the pivot element is neverthe smallest element of the array.◮If we use median-of-three partitioning, then the partitionelement can never be the smallest element in the array.Average Case Analysis◮Assume the partition element is chosen randomly.◮Assume all k ∈ {1, 2 . . . , n} are equally likely (each hasprobability1n)◮The average case running time is the solution toT(n) = n + 1 +1nnXk=1(T (k − 1) + T(n − k))along with T(0) = T(1) = 1.◮Can simplify:T(n) = n + 1 +2nnXk=1T(k − 1)Average Case Analysis (cont.)◮We can eliminate the sum by multiplying both sides by nand subtracting the formula for T(n-1).nT(n) − (n − 1)T(n − 1) = n(n + 1) − (n − 1)n + 2T(n − 1)⇒ nT(n) = (n + 1)T(n − 1) + 2n◮The solution to this is in Θ(n log n).◮In fact, the exact solution is more like 1.39n log n.◮This means that the average case is only about 40% slowerthan the best case!Duplicate Keys◮Quicksort can be inefficient in the case that the arraycontains many duplicate keys.◮In fact, if the array consists entirely of records withidentical keys, our implementation so far will still performthe same amount of work.◮The easiest way to handle this is to do three-waypartitioning.◮Instead of splitting the array into only two pieces, we havea third piece consisting of the elements equal to thepartition element.Small Subarrays◮Another way in which quicksort, as well as other recursivealgorithms, can be optimized is by sortingsmall subarraysdirectly using insertion sort.◮Empirically, subarrays of approximately 10 elements orsmaller should be sorted directly.◮An even better approach is to simply ignore the smallsubarrays and then insertion sort the entire array oncequick sort has finished.◮There will be ⌈n/10⌉ calls to insSort(), each on asubarray of 10 elements.Function calls an d data structures◮While playing Hanoi Towers with n = 30, you realize youhave to read Chapter 13; stop playing and start reading◮At 12:30pm, you stop reading and go eat lunch◮During lunch, somebody calls your cell phone; you stopeating and answer the call◮You talk for a minute, then hang up and resume eating◮Lunch is over; resume reading Chapter 13◮Done with Chapter 13; go back play Hanoi TowersEvery time you call the “functions” readCh13(), lunch(),answerCell(), you save the current status to remember whatyou were doing before calling that function.Q.: What’s the data structure for saving the current statuses?A.: The stack.Stacks◮last in, first out (LIFO) data structures◮The last status saved will be the first resumed◮Computers use a stack for saving the current status beforerunning a called function:◮CPU registers (super fast, small memory areas)◮Address of the instruction being executed (ProgramCounter – PC)◮variables of the caller function1◮Whenever a function is