CORNELL CS 664 - Lecture #9: Binary labeling problems and their applications

Unformatted text preview:

CS664 Lecture #9: Binary labeling problems and their applicationsAnnouncementsRecapGraph constructions-t Graph Cuts for General Binary LabelingWhy should anyone care about binary labeling problems?Application 1: shape reconstructionVoxel algorithm for volume intersectionVoxel algorithm for volume intersectionSilhouette intersectionProblem formalizationEnergy functionWhat smoothness term?Examples on real dataVisualizationAnother exampleFly-throughsApplication 2: medical image segmentations-t Graph Cut for General Binary Labelings-t Graph Cut for General Binary LabelingInteractive Object ExtractionInteractive Object ExtractionApplication 3: object “cut and paste” Photomontage ProblemFramework Graph Cuts Boykov and Jolly (2001) Iterated Graph Cut Iterated Graph Cuts Colour Model Coherence Model Moderately straightforward examples Difficult ExamplesEvaluation – Labelled DatabaseCS664 Lecture #9: Binary labeling problems and their applicationsSome material taken from:Carsten Rother et al, Microsoft Researchhttp://research.microsoft.com/vision/cambridge/i3l/segmentation/GrabCut.htmOlga Veksler, University of Western Ontariohttp://www.csd.uwo.ca/faculty/olga/2Announcements PS1 is on the web, and debugged (?)– Due on 10/6 Quiz 2 on Tuesday 9/27– Coverage through last lecture3 We want to minimize the energy E(f) Binary labeling problem – Also known as Ising model (1924)Recap4Graph construction Binary labeling via graph cuts example: •λ = 3• DP(0) = 1 and DP(1) = 4 whenever I(p) = 0• DP(0) = 3 and DP(1) = 2 whenever I(p) = 101p rqsblackwhite333324114235s-t Graph Cuts forGeneral Binary Labeling Binary Labeling problem:– We have 2 labels, let’s call them s and t (associated with the source and sink terminals, respectively)– We have a set of sites (image pixels, volume voxels, so on) to which we wish to assign these labels– Each site p prefers one label more than another– We wish to find a labeling L (assignment of labels s and t to sites) such that• Each site is assigned a label it prefers, as much as possible• Nearby pixels tend to have similar labels6Why should anyone care about binary labeling problems? Remarkably powerful and common– Now that we can solve them fast!– You can use graph cuts to evaluate simulated annealing, and also to determine if the minimum is actually what you want• Pretty good, but not quite (why?) 3 applications today:– 3D shape reconstruction (CVPR 2000)– Medical image segmentation (ICCV 2001)– Object “cut and paste” (SIGGRAPH 2004)7Application 1: shape reconstruction Snow, Viola & Zabih, CVPR00– Create a 3D shape from multiple cameras Generalization of “image differencing”– Take two consecutive images– Subtract and threshold Easy to build a simple motion detector– Common in “security webcams”– Where does this fail? Consider an object surrounded by cameras– Or, on a turntable8Voxel algorithm for volume intersection Color voxel gray if on silhouette in every image9Voxel algorithm for volume intersectionhole Color voxel gray if on silhouette in every image10Silhouette intersection Early hard decisions are a problem– Errors in silhouette identification– Can lead to “holes” Can we add spatial smoothness with energy minimization? More importantly, can we do it fast?11Problem formalization We are given a set of voxels p∈P which must be assigned binary labels fp– 1 if filled, 0 if empty A pixel in an image corresponds to a set of voxels in a line– If there is a large change in intensity (versus background image), these voxels “prefer” to be 1’s. Similarly for 0’s. Note what we don’t represent:– Surfaces, photoconsistency, etc.12Energy function The cost for the voxel p to have the label fpwill be written Dp(fp) We seek the labeling f that minimizesfor some smoothness term Esmooth(f))()()( fEfDfEsmoothppp∑+=13What smoothness term? It is easy to imagine lots of clever smoothness terms– That we have no idea how to minimize! Here is a natural one where we can efficiently find the global minimum!][),(),()(,qpqpNqpqpsmoothffffVffVfE≠==∑∈T14Examples on real data15Visualization16Another exampleConventional Energy minimization17Fly-throughs Synthetic example (cylinder)– silhouette intersection reconstruction– graph cuts reconstruction Sara– graph cuts reconstruction Paul– silhouette intersection reconstruction– graph cuts reconstruction18Application 2: medical image segmentation Boykov & Jolly, ICCV01– Interactive tool to segment objects19s-t Graph Cut forGeneral Binary Labeling For the fax image problem, we minimized:()()()∑∑∈≠+=NpqqpppPLLLDLETλ More general energy function: ()()()∑∑∈≠+=NpqqppqppPLLwLDLET¾ The weight of discontinuity is wpqand it depends on a particular pair of pixels p and q.In grayscale (or color) images, discontinuities between similarly colored pixels cost more than discontinuities between differently colored pixels; object boundaries tend to coincide with intensity boundaries20s-t Graph Cut forGeneral Binary Labeling How does our graph construction change?¾ Cost of t-links does not change¾ Instead of λ, make the weight of n-link between pixels p and qto be wpqstp rquDp(t)Dp(s)Du(s)Du(t)wqrwruwpq21sthard constrainthard constraintInteractive Object Extraction 2 labels: background (blue), object (red) User clicks on a few pixels with blue and red paintbrushes Data term¾ DP(blue) = infinity (that is a very large number) if the user clicked on pixel p with a red paintbrush9 Hard constraint: pixel p must be in the background ¾ DP(red) = infinity (that is a very large number) if the user clicked on pixel p with a blue paintbrush9 Hard constraint: pixel p must be in the object¾ Otherwise DP(red) = DP(blue) = 022n-linkssta cuthard constrainthard constraint Prior term¾ Object boundaries tend to have sharp intensity edge¾ Encourage cutting around sharp intensity edge¾ Make wpqcheaper for larger intensity difference between pand qInteractive Object Extraction2324Application 3: object “cut and paste” Rother, Kolmogorov, Blake SIGGRAPH04PhotomontagePhotomontagePhotomontageGrabCut – Interactive Foreground Extraction 1ProblemProblemProblemGrabCut – Interactive Foreground Extraction 2Fast & Accurate ?FrameworkFrameworkFrameworkInput: Image Output: Segmentation Parameters:


View Full Document
Download Lecture #9: Binary labeling problems and their applications
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Lecture #9: Binary labeling problems and their applications and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Lecture #9: Binary labeling problems and their applications 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?