Greedy AlgorithmsA short list of categoriesOptimization problemsExample: Counting moneyA failure of the greedy algorithmA scheduling problemAnother approachAn optimum solutionHuffman encodingMinimum spanning treeTraveling salesmanAnalysisOther greedy algorithmsDijkstra’s shortest-path algorithmAnalysis of Dijkstra’s algorithm IAnalysis of Dijkstra’s algorithm IIConnecting wiresCollecting coinsThe End1Greedy Algorithms22A short list of categoriesAlgorithm types we will consider include:Simple recursive algorithmsBacktracking algorithmsDivide and conquer algorithmsDynamic programming algorithmsGreedy algorithmsBranch and bound algorithmsBrute force algorithmsRandomized algorithms33Optimization problemsAn optimization problem is one in which you want to find, not just a solution, but the best solutionA “greedy algorithm” sometimes works well for optimization problemsA greedy algorithm works in phases. At each phase:You take the best you can get right now, without regard for future consequencesYou hope that by choosing a local optimum at each step, you will end up at a global optimum44Example: Counting moneySuppose you want to count out a certain amount of money, using the fewest possible bills and coinsA greedy algorithm would do this would be:At each step, take the largest possible bill or coin that does not overshootExample: To make $6.39, you can choose:a $5 billa $1 bill, to make $6a 25¢ coin, to make $6.25A 10¢ coin, to make $6.35four 1¢ coins, to make $6.39For US money, the greedy algorithm always gives the optimum solution55A failure of the greedy algorithmIn some (fictional) monetary system, “krons” come in 1 kron, 7 kron, and 10 kron coinsUsing a greedy algorithm to count out 15 krons, you would getA 10 kron pieceFive 1 kron pieces, for a total of 15 kronsThis requires six coinsA better solution would be to use two 7 kron pieces and one 1 kron pieceThis only requires three coinsThe greedy algorithm results in a solution, but not in an optimal solution66A scheduling problemYou have to run nine jobs, with running times of 3, 5, 6, 10, 11, 14, 15, 18, and 20 minutesYou have three processors on which you can run these jobsYou decide to do the longest-running jobs first, on whatever processor is available201815 141110653P1P2P3Time to completion: 18 + 11 + 6 = 35 minutesThis solution isn’t bad, but we might be able to do better77Another approachWhat would be the result if you ran the shortest job first?Again, the running times are 3, 5, 6, 10, 11, 14, 15, 18, and 20 minutesThat wasn’t such a good idea; time to completion is now6 + 14 + 20 = 40 minutesNote, however, that the greedy algorithm itself is fastAll we had to do at each stage was pick the minimum or maximum201815141110653P1P2P388An optimum solutionThis solution is clearly optimal (why?)Clearly, there are other optimal solutions (why?)How do we find such a solution?One way: Try all possible assignments of jobs to processorsUnfortunately, this approach can take exponential timeBetter solutions do exist:201815141110 653P1P2P399Huffman encodingThe Huffman encoding algorithm is a greedy algorithmYou always pick the two smallest numbers to combineAverage bits/char:0.22*2 + 0.12*3 +0.24*2 + 0.06*4 +0.27*2 + 0.09*4= 2.42The Huffman algorithm finds an optimal solution22 12 24 6 27 9 A B C D E F15274654100A=00B=100C=01D=1010E=11F=10111010Minimum spanning treeA minimum spanning tree is a least-cost subset of the edges of a graph that connects all the nodesStart by picking any node and adding it to the treeRepeatedly: Pick any least-cost edge from a node in the tree to a node not in the tree, and add the edge and new node to the treeStop when all nodes have been added to the treeThe result is a least-cost (3+3+2+2+2=12) spanning treeIf you think some other edge should be in the spanning tree:Try adding that edgeNote that the edge is part of a cycleTo break the cycle, you must remove the edge with the greatest costThis will be the edge you just added12345633332224441111Traveling salesmanA salesman must visit every city (starting from city A), and wants to cover the least possible distanceHe can revisit a city (and reuse a road) if necessaryHe does this by using a greedy algorithm: He goes to the next nearest city from wherever he isFrom A he goes to BFrom B he goes to DThis is not going to result in a shortest path!The best result he can get now will be ABDBCE, at a cost of 16An actual least-cost path from A is ADBCE, at a cost of 14EA B CD23 344 41212AnalysisA greedy algorithm typically makes (approximately) n choices for a problem of size n(The first or last choice may be forced)Hence the expected running time is:O(n * O(choice(n))), where choice(n) is making a choice among n objectsCounting: Must find largest useable coin from among k sizes of coin (k is a constant), an O(k)=O(1) operation;Therefore, coin counting is (n)Huffman: Must sort n values before making n choicesTherefore, Huffman is O(n log n) + O(n) = O(n log n)Minimum spanning tree: At each new node, must include new edges and keep them sorted, which is O(n log n) overallTherefore, MST is O(n log n) + O(n) = O(n log n)1313Other greedy algorithmsDijkstra’s algorithm for finding the shortest path in a graphAlways takes the shortest edge connecting a known node to an unknown nodeKruskal’s algorithm for finding a minimum-cost spanning treeAlways tries the lowest-cost remaining edgePrim’s algorithm for finding a minimum-cost spanning treeAlways takes the lowest-cost edge between nodes in the spanning tree and nodes not yet in the spanning tree1414Dijkstra’s shortest-path algorithmDijkstra’s algorithm finds the shortest paths from a given node to all other nodes in a graphInitially, Mark the given node as known (path length is zero)For each out-edge, set the distance in each neighboring node equal to the cost (length) of the out-edge, and set its predecessor to the initially given nodeRepeatedly (until all nodes are known),Find an unknown node containing the smallest distanceMark the new node as knownFor each node adjacent to the new node, examine its neighbors to see whether their estimated distance can be reduced (distance to known node plus cost of
View Full Document