IE172 – Algorithms in Systems EngineeringLecture 21Pietro BelottiDepartment of Industrial & Systems EngineeringLehigh UniversityMarch 11, 2009Next time: Read Chapter 25 up to section 25.2The Minimum Spanning Tree (MST) problemA classical problem. Given:◮a s et of cities and a set of potential roads◮(i.e. a road between two cities can only be built if there is aviable path between them)◮Each road connects two and only two cities◮Constructing road from city u to city v costs wuvTheobjective: construct roads such that◮Everyone is connectedi.e. There is a path betwee n any two cities in the set◮The total construction cost is minimumSpanning TreeWe mod el the problem as a graph problem.◮G = (V, E) is an undirected graph◮Weights w : E → R|E|, i.e., wuv∀(u, v) ∈ EFind T ⊂ E such that◮T connects all vertices◮The w eight w(T)def=X(u,v)∈Twuvis minimized◮The notation T is not a coincidence.◮The set of edges T will form a tree. (Why?)◮T is known as a minimum spanning tree (MST) of GHow to Buil d It?◮Let T be the empty set◮Add to T keeping the following loop invariant:T is always a subset of an MST◮An edge {u, v} ∈ E is safe for T if T ∪ {{u, v}} is also asubset o f an MST◮The initial T = ∅ is safe◮In the i-th iteration of the loop, we add a safe edge⇒ Quickly detect and add safe edge s.GENERIC-MST(V, E, w)1 T ← ∅2 while T is not a spanning tree3 do find {u, v} ∈ E that is safe for T4 T ← T ∪ {{u, v}}5 return TFinding Safe Edges◮How do y ou ensure that T is always a subset of an MST?◮Or equivalently, th at an e dge {u, v} ∈ E is safe for T ifT ∪ {{u, v}} is also a subset of an MST?A subset of an MST is simply a subgraph o f G that is a tree.How do I know if {u, v} is safe? (Intuition)◮Let S ⊂ V be any set of vertices that includes u but not v◮In any MST there must be at least one edge that conn ectsS to V \ S, so let’s make thegreedy choice of choosingthe one with the minimum weightMore definitions for graph G = (V, E)◮A cut (S, V \ S) is a partition of the vertices into disjointsets S and V \ S◮An edge {u, v} ∈ E crosses cut (S, V \ S) is one endpoint isin S and the other is in V \ S◮A cut is said to respect a set of edges T ⊆ E if no edge in Tcrosse s the cutMST TheoremLet T be a subset of some MST, let (S, V \ S) be a cut that re-spects T, and let {u, v} be the minimum weight edge crossing(S, V \ S). Then {u, v} is safe for TKruskal’s Algorithm◮Start with each vertex being its own component◮Merge two compon ents into one by choosing the light1edge that connects them◮Scan th e set o f edg es in increasing order o f weight◮Use an abstract “disjoint sets” data structure to check if anedge conn ects d ifferent vertices in different se ts.1i.e., the cheapest.Kruskal’s AlgorithmKRUSKAL(V, E, w)1 T ← ∅2 for each v in V3 do MAKE-SET(v)4 SORT(E, w)5 for each {u, v} in (sorted) E6 do if FIND-SET(u) 6= FIND-SET(v)7 then T ← T ∪ {{u, v}}8 UNION(u, v)9 return TAnalysisLet T (X ) be the running time of the method X .TaskRunning TimeInitialize T O(1)First “for” loop|V|T (MAKE-SET)Sort EO(|E| lg |E|)Second “for” loopO(|E|)(T (FIND-SET + UNION)We Skipped That Chapter!◮If we use a clever data structure for FIND-SET and UNION,the running time can go to α(m, n), where m is the tot alnumber of operations, and n is the number of unions.◮α(m, n) is th e inverse of the Ackerman function, which is aslowly g rowing function.◮α(m, n) ≤ 4 for all practical purposes◮In this case, we have that the operations t ake α(|E|, |V|)Kruskal Analysis◮Also, you should know that α(|E|, |V|) = O(lg |V|)◮Finally, note that|E| ≤ |V|2⇒ lg |E| = O(2 lg |V|) = O(lg |V|)⇒ The running time for Kruskal’s Algorithm is O(|E| lg |V|).◮If the edges are already sorted, it runs in O(|E|α(|E|, |V|)),which is essentially linearPrim’s Algorithm◮Builds o n e tree, so T is always a tree◮Let VTbe the set of vert ices on which T is incident◮Start from an arbitrary root r◮At each st ep find a light edge crossing the cut (VT, V \ VT)Main Question for Prim:How do we find a light edge crossing the cut quickly?Prim’s AlgorithmThe answer:◮Use a priority queue!e.g. implemented as a heap◮Each object in the queue is a vertex in V \ VT(a vertex thatmight be linked to ou r MST)◮The key of v is the minimum weight of any edge {u, v}such that u ∈ VT.◮The key of v is ∞ if v is not adjacent to any vertices in VTPrim’s Algorithm◮Start from an (arbitrary) vertex (the root r)◮Keep track of the parent π[v] of every ve r tex v. (π[r] = NIL).◮As the algorithm processes, T = {{v, π[v]} | v ∈ V \ {r} \ Q}◮At t ermination, VT= V ⇒ Q = ∅, so MST isT = {(v, π[v]) | v ∈ V \ {r}}.Pseudocode for PrimPRIM(V, E, w, r)1 Q ← ∅2 for each u ∈ V3 do key[u] ← ∞4 π[u] ← NIL5 INSERT(Q, u)6 key[r] = 07 while Q 6= ∅8 do u ← EXTRACT-MIN(Q)9 for each v ∈ Adj[u]10 do if v ∈ Q and wuv< key[v]11 then π[v] ← u12 key[v] = wuvShortest Paths—Definitions◮For the next few lectures, w e will have a directed graphG = (V, A), and a weight function w : A → R|A|.◮The w eight o f a path P = (v0, v1, . . . vk) is simply theweight of the arcs taken on the sequence of nodes:w(P) =kXi=1wvi−1,vi.◮We are interested in finding the shortest path from s to t,which we will denote δ(s, t).◮We use the convention that δ(s, t) = ∞ if t h ere is no pathfrom s to t in GShortest path subgraph◮Consider all shortest (directed) paths from a given node sto any othe r node of V◮This set of paths forms a subgraph of G◮Why? Because each arc of G may be in zero or moreshortest paths◮What does this subgraph look like? Can it have cycles?st1t2t3c1c2c3c4Shortest path graph◮If it had a directed cycle it w ould cost nothing to run onthat cycle⇒ Shortest paths from a single node s are organized as a tree◮Indeed these subgraphs are called shortest path trees◮Many algorithms work like a generalization of BFS toweighted graphs.Shortest Path Variants◮Single-Source: Find the shortest path from s ∈ V to everyvertex v ∈ V◮Single-Destination: Find the shorte st path from everyvertex v ∈ V to agiven destination vertex t ∈ V◮Single-Pair: Find the shor test path from given s ∈ V togiven t ∈ V. There is now way known that is bette r (in theworst case) that solving the single-source version.◮All-Pairs: Find th e shortest path from every u ∈ V to everyvertex v ∈