CS664 Lecture #24: Curve evolution, differential geometry, level setsAnnouncementsCurve evolution overviewImplementing snake evolutionCurve evolution of snakesArc length parameterizationReparameterizationForces acting on the curveGeneralized curve evolutionCurvature flowLocal coordinate systemGeometric flowsGeodesic active contoursLevel setsLevel set surfacesExampleExampleLevel set evolutionSpeedup: narrow bandState of the artBetter optimization?Can we do better?Geo-cutsSegmentation via min cut?Segmentation via min cutWu and LeahyNested cuts (Veksler)PropertiesCS664 Lecture #24: Curve evolution, differential geometry, level setsSome material taken from:Yuri Boykov & Olga Veksler, Western OntarioPushmeet Kohli, Oxford BrookesDonald Tanguay, HP LabsNikos Paragios, Ecole Centrale de Parishttp://cermics.enpc.fr/~paragios/tutorial.ppt2Announcements 1-paragraph final project description due by email on 11/23 Final quiz will be on 11/29 PS3 will be out soon, due Friday 12/2 Final project will be due Thursday 12/153Curve evolution overview We will consider “virtual time” within a single image– Given an initial curve, move it to line up with image features (i.e., edges)– Keep the curve reasonably smooth Notation: we will write a curve as C(p)=(x(p),y(p)), p∈[0,1]– Note that we take total derivatives w.r.t. p4Implementing snake evolution Discrete:– Stored as vertices– Each vertex is moved iteratively Continuous:– Stored as particles– Each particle is moved– New particles arecomputed (interpolation)5Curve evolution of snakes We can simplify our energy function to Calculus of variations says that at a local minimum of the energy we have– Justification: small changes don’t change this– We can minimize this via gradient descent– Many ugly issues with this…6Arc length parameterization If we replaced p by φ, where φ(r)=p, r∈[c,d], first term in energy would become– Second term is even worse! Natural parameterization is in terms of arc length (distance along curve)– s(p) is the distance from the origin to p:7Reparameterization We can use s to reparameterize the curve– Instead of p∈[0,1] use s∈[0,L] Many nice properties, e.g.– Unit velocity |Cs|=1– Curve and its derivatives are invariant to rotation and translation– Intrinsic properties of curves don’t depend upon parameterizations, only on curves• With this parameterization, curve and all derivatives are intrinsic10pLCdp=∫8Forces acting on the curve Instead of an energy function E that we minimize, we think about a force acting on the curve over time– Force can do gradient descent over E9Generalized curve evolution Natural forces involve internal and external terms– As usual, external are easy• Roll towards edges in the image– Internal ones have many interesting variants Curvature flow – Smooths out curves (geometric heat equation)– Each particle moves perpendicularly to the curve with speed proportional to curvature10Curvature flowlargepositivemotionsmallnegativemotion11Local coordinate systemTangent T=(xs,ys)Normal N=(-ys,xs)Osculating circle (radius = 1/κ)12Geometric flows Natural coordinates are defined by N,T– Ct= VNN + VT T– We can ignore the second term since it only affects the particles, not the curve For a geometric flow, β = VNis independent of the parameterization– Geometric heat equation has β = κ– This takes any simple (non-self intersecting) curve first into a convex shape, then into a circular point that vanishes– Time to vanish is area enclosed13Geodesic active contours An elegant solution which is very close to the original snake energy is simply– Where did the internal energy go??– Replaced by minimizing curve length• Geodesic, where g controls the local metric Curve evolution to minimize E is14Level sets Curve evolution is great. But:– Topology changes are a major problem– Also, numerical issues and stopping conditions• Particles tend to colide, and solutions oscillate Level sets address these beautifully Basic idea: implicit function for curve– Think of a topological map of a park– There are curves drawn at a fixed height• I.e., the points where height = 1 mile– We will use this surface to evolve the curve15Level set surfaces To evolve a curve C, first compute a surface u such that C is level set of u– Obviously, not unique– Standard choice: signed distance function• u(x,y) = distance to nearest point in C• Negative if inside of C Instead of evolving the curve, we evolve the surface!– Sometimes called “contour propagation”, “interface tracking”, etc.16Example),(1yxuzt==}0),(:),{(== yxuyxC(implicit contour representation)17Example-0.80.20.50.70.40.6-0.2-1.7-0.6-0.8-0.4-0.5),(jiijyxuu=C18Level set evolution Instead of evolving C we evolve u Curve evolution: Level set evolution: Amazing fact: if you evolve u by ut, everylevel set of u will evolve according to Ct19Speedup: narrow band20State of the art Almost everyone who does curve evolution these days uses level sets– Handles changes in topology– Numerical stability– Natural generalization to higher dimensions• Evolving surfaces (“minimal surfaces”) Typically add some sort of constant force (“balloon pressure”) plus curvature It would be great to have a better way to incorporate shape21Better optimization? Geodesic active contours are very elegant– Use a local (Riemannian) metric• Depends on image features But level sets only yield a local optimum!Riemannian metric(space varying, tensor D(p))Euclidian metric(constant)ABABdistance mapdistance map22Can we do better? Shortest paths are 1D only, but it seems like something graph-like could be used… Cost of a cut is “length” (“area”) of the corresponding curve (surface)C∑∈=CeeC ||||||23Geo-cuts It is possible to choose the edge weights so that the cost of a cut corresponds to the length (area)– Under arbitrary Riemannian metric, or beyond You can find a minimal surface which is actually minimal!– Under given boundary conditions– As in binary segmentation from graph cuts24Segmentation via min cut? Why can’t we use min cut to directly segment an image?– Weights between edges are “affinities”• I.e., decreasing function of |Ip–Iq|– Where do the source and sink go? Two answers– Use variant on min cut where there is no source and