Lecture Notes CMSC 251CMSC 251: Algorithms1Spring 1998Dave MountLecture 1: Course Introduction(Tuesday, Jan 27, 1998)Read: Course syllabus and Chapter 1 in CLR (Cormen, Leiserson, and Rivest).What is algorithm design? Our text defines an algorithm to be any well-defined computational procedurethat takes some values as input and produces some values as output. Like a cooking recipe, an algorithmprovides a step-by-step method for solving a computational problem.A good understanding of algorithms is essential for a good understanding of the most basic element ofcomputer science: programming. Unlike a program, an algorithm is a mathematical entity, which isindependent of a specific programming language, machine, or compiler. Thus, in some sense, algorithmdesign is all about the mathematical theory behind the design of good programs.Why study algorithm design? There are many facets to good program design. Good algorithm design isone of them (and an important one). To be really complete algorithm designer, it is important to beaware of programming and machine issues as well. In any important programming project there aretwo major types of issues, macro issues and micro issues.Macro issues involve elements such as how does one coordinate the efforts of many programmersworking on a single piece of software, and how does one establish that a complex programming systemsatisfies its various requirements. These macro issues are the primary subject of courses on softwareengineering.A great deal of the programming effort on most complex software systems consists of elements whoseprogramming is fairly mundane (input and output, data conversion, error checking, report generation).However, there is often a small critical portion of the software, which may involve only tens to hundredsof lines of code, but where the great majority of computational time is spent. (Or as the old adage goes:80% of the execution time takes place in 20% of the code.) The micro issues in programming involvehow best to deal with these small critical sections.It may be very important for the success of the overall project that these sections of code be writtenin the most efficient manner possible. An unfortunately common approach to this problem is to firstdesign an inefficient algorithm and data structure to solve the problem, and then take this poor designand attempt to fine-tune its performance by applying clever coding tricks or by implementing it on themost expensive and fastest machines around to boost performance as much as possible. The problem isthat if the underlying design is bad, then often no amount of fine-tuning is going to make a substantialdifference.As an example, I know of a programmer who was working at Boeing on their virtual reality system forthe 777 project. The system was running unacceptably slowly in spite of the efforts of a large team ofprogrammers and the biggest supercomputer available. A new programmer was hired to the team, andhis first question was on the basic algorithms and data structures used by the system. It turns out thatthe system was based on rendering hundreds of millions of polygonal elements, most of which were1Copyright, David M. Mount, 1998, Dept. of Computer Science, University of Maryland, College Park, MD, 20742. These lecturenotes were prepared by David Mount for the course CMSC 251, Algorithms, at the University of Maryland, College Park. Permissionto use, copy, modify, and distribute these notes for educational purposes and without fee is hereby granted, provided that this copyrightnotice appear in all copies.1Lecture Notes CMSC 251invisible at any point in time. Recognizing this source of inefficiency, he redesigned the algorithms anddata structures, recoded the inner loops, so that the algorithm concentrated its efforts on eliminatingmany invisible elements, and just drawing the few thousand visible elements. In a matter of two weekshe had a system that ran faster on his office workstation, than the one running on the supercomputer.This may seem like a simple insight, but it is remarkable how many times the clever efforts of a singleclear-sighted person can eclipse the efforts of larger groups, who are not paying attention to the basicprinciple that we will stress in this course:Before you implement, first be sure you have a good design.This course is all about how to design good algorithms. Because the lesson cannot be taught in justone course, there are a number of companion courses that are important as well. CMSC 420 deals withhow to design good data structures. This is not really an independent issue, because most of the fastestalgorithms are fast because they use fast data structures, and vice versa. CMSC 451 is the advancedversion of this course, which teaches other advanced elements of algorithm design. In fact, many of thecourses in the computer science department deal with efficient algorithms and data structures, but justas they apply to various applications: compilers, operating systems, databases, artificial intelligence,computer graphics and vision, etc. Thus, a good understanding of algorithm design is a central elementto a good understanding of computer science and good programming.Implementation Issues: One of the elements that we will focus on in this course is to try to study algorithmsas pure mathematical objects, and so ignore issues such as programming language, machine, and op-erating system. This has the advantage of clearing away the messy details that affect implementation.But these details may be very important.For example, an important fact of current processor technology is that of locality of reference. Fre-quently accessed data can be stored in registers or cache memory. Our mathematical analyses willusually ignore these issues. But a good algorithm designer can work within the realm of mathemat-ics, but still keep an open eye to implementation issues down the line that will be important for finalimplementation. For example, we will study three fast sorting algorithms this semester, heap-sort,merge-sort, and quick-sort. From our mathematical analysis, all have equal running times. However,among the three (barring any extra considerations) quicksort is the fastest on virtually all modern ma-chines. Why? It is the best from the perspective of locality of reference. However, the difference istypically small (perhaps 10–20% difference in running time).Thus this course is not the last word in good program design, and in fact it is perhaps