CS664 Lecture #11: Graph cuts for more than two labelsAnnouncementsRecapLocal improvement methodLocal improvement vs. Graph cutsMove examplesThe swap move algorithmThe expansion move algorithmAlgorithm propertiesVery large neighborhood search Swap move algorithmStart with an arbitrary labelingCompute min cut on subgraphEnd of iterationExpansion move algorithmExpansion move algorithmPotts model error boundExpansion moves in actionBinary sub-problemExpansion move energyGraph cuts solutionRegular choices of VWhat V’s are metrics?Old methods: How fast do you want the wrong answer?Statistical performanceExpansion move applicationsApplication: texture synthesisGraphcuts video texturesAnother exampleInteractive Digital PhotomontageCS664 Lecture #11: Graph cuts for more than two labelsSome material taken from:Yuri Boykov, University of Western Ontariohttp://www.csd.uwo.ca/faculty/yuri/Aseem Agarwala, University of Washingtonhttp://www.cs.washington.edu/homes/aseem2Announcements PS 1 due in a week Guest lecture on Tuesday by Ashish Raj– Some vision problems in MR reconstruction3Recap Linear potentials can be solved exactly Potts model can be turned into the multiway cut problem4Local improvement method Subproblem: pick a pixel, find the label that minimizes E, repeat– Minimize restricted version of E (line search)– Computes a local minimum5 Continuous vs. discrete– No floating point with graph cuts Local min in line search vs. global min Minimize over a line vs. hypersurface– Containing O(2n) candidates Local minimum: weak vs. strong– 2-approximation for the Potts model– Within 0.15% of global min!Local improvement vs. Graph cuts6Move examplesStarting pointRed-blue swap moveGreen expansion move7The swap move algorithm1. Start with an arbitrary labeling2. Cycle through every label pair (α,β)2.1 Find the lowest E labeling within a single αβ-swap2.2 Go there if it’s lower E than the current labeling3. If E did not decrease in the cycle, done Otherwise, go to step 28The expansion move algorithm1. Start with an arbitrary labeling2. Cycle through every label α2.1 Find the lowest E labeling within a single α-expansion2.2 Go there if it’s lower E than the current labeling3. If E did not decrease in the cycle, done Otherwise, go to step 29Algorithm properties Graph cuts (only) used in key step 2.1– On a binary sub-problem, as we’ll see In a cycle the energy doesn’t increase– Convergence in O(n) cycles• In practice, termination occurs in a few cycles When the algorithms converge, the resulting labeling is a local minimum– Even when allowing an arbitrary swap move or expansion move10Very large neighborhood search  A local minimum with respect to these moves is the best answer in a very large neighborhood– For example, there are O(k 2n) labelings within a single expansion move– Starting at an arbitrary labeling, you can get to the global minimum in k expansion moves11Swap move algorithm Let’s look at how we compute the best swap move– It’s basically just the two-label case we already know about Quite simple for the Potts model12Start with an arbitrary labelingABC13Compute min cut on subgraph The cost of cutting an n-link clearly depends on A and BAB14End of iteration Return the other nodes to the graphABC15Expansion move algorithm Most powerful graph cut method– Widely used in practice Interesting theoretical properties also– 2-approximation for Potts model16Expansion move algorithm– Find green expansion move that most decreases E• Move there, then find the best blue expansion move, etc• Done when no α-expansion move decreases the energy, for any label αInput labeling fGreen expansion move from f17fˆlocal minimumoptimal solution*fστσαστ⎩∉ Apfp⎨⎧∈=ApfˆααααααααααααααA()()()*AAAfEfEfˆE ≤≤αSumming up over all labels:()()()()***2ˆfEfEfEfE ≤+≤∂αff aˆ()()αfEfE ≤ˆ⇒ααααααστσαστλλγγγυυυφφφωωPotts model error bound18Expansion moves in actioninitial solution-expansion-expansion-expansion-expansion-expansion-expansion-expansionFor each move we choose expansion that gives the largest decrease in the energy: binary energy minimization subproblem19Binary sub-problemInput labeling Expansion move Binary image20Expansion move energyGoal: find the binary image with lowest energyBinary image energy is a restriction of EDepends on f,α21Graph cuts solution This can be done as long as V has a specific form (works for arbitrary D) Regularity constraint [KZ PAMI ’04]– Can find cheapest α-expansion from f if22Regular choices of V Suppose that V is a metric Then what?23What V’s are metrics? Important examples– Potts model– L1 distance– Truncated L1 distance– NOT the L2 distance (with or without truncation) There is an easy way to check the triangle inequality24Old methods: How fast do you want the wrong answer?Right answers Fast (correlation)Slow (annealing)FastSlowGraph cuts25Statistical performance26Expansion move applications Lots of them, in vision and in graphics We’ll look at a few really “cute” examples– I.e., from SIGGRAPH– Texture synthesis (“graphcut textures”)– Photomontage– Panoramic video textures Then two important ones from vision– Stereo with sloped and curved surfaces• Via the Potts model!– Multi-camera stereo27Application: texture synthesisABCDEFGHIJABGDCFHIJE“Graphcut textures”(Kwatra, Schodl, Essa, Bobick SIGGRAPH2003)http://www.cc.gatech.edu/cpl/projects/graphcuttextures/28Graphcuts video texturesShort video clip21Long video clipa cut29Another exampleoriginal short clipsynthetic infinite textureInteractive Digital PhotomontageAseem Agarwala, Mira Dontcheva, Maneesh Agrawala, Steven Drucker, Alex Colburn, Brian Curless, David Salesin, Michael CohenUniversity of Washington & Microsoft Researchactualset of originalsphotomontageperceivedSource images Brush strokes Computed labelingCompositeBrush strokes Computed labelingImage objective0 if red∞ otherwise0 for any labelDemoscourtesy of P.