1Extracting Subimages of an Unknown Category from a Set of Images Sinisa Todorovic and Narendra AhujaBeckman Institute, UIUCPresented by Tingfan WuObjectiveClustersGeneral StepsTraining ImagesRandom segmentsfeature vectorsUnseen imageC3C2C1ModelsFt1=(x1,x2….xn)Ft2=(x1,x2….xn)Ft3=(x1,x2….xn)Ft4=(x1,x2….xn)feature vectorsF1=(x1,x2….xn)F2=(x1,x2….xn)F3=(x1,x2….xn)F4=(x1,x2….xn)• Varieties– Segmetation Methods– Feature Spaces– Clustering Methods= C1Segment out all the cars….fused tree model for carsUnseen imageTraining imagesSegmented CarsSegmentation TreesOverviewOverviewMultiscale Seg.Segment out all the cars….fused tree model for carsUnseen imageTraining imagesSegmented CarsSegmentation TreesMultiscaleMultiscaleSegmentation TreeSegmentation TreeFeature Extraction = Image SegmentationMultiscale Segmentation TreeRegion Descriptor on Tree NodeAttr(Node) = Description of the regionWhat are good region descriptors?• Photometric– Gray level• Geometric (rotation invariant)–Area–C.M.– Boundary Shape Histogram•Hybrid – Salient descriptor• Topology– Recursive containment of regions(¹v;¾2v)hv(1:::K )(av)(xv;yv)(©v)hv(1)hv(2)hv(8)Can be rotation invarianthv(3)Salient Descriptor for a Region• An outstanding region among siblings?– Brighter/darker?– Noisier /more homogenous– Larger/Smaller– Higher/lower entropy on boundary shape• Empirical result: best λ=0.5PhotometricGeometrichv(1)hv(2)hv(8)...Salience Contract Flow(microview)Average Direction and Magnitude¡!©v¡!wv=d2++++++Salience Contract Flow(macroview)Match salience contract flow~¡!©1¼¡!©2Store Regional Descriptor on TreenodePhotometric GeometricSalientSegment out all the cars….fused tree model for carsUnseen imageTraining imagesSegmented CarsSegmentation TreesMaximal CommonMaximal CommonSubtreeSubtreeMatching MatchingSegment out all the cars….fused tree model for carsUnseen imageTraining imagesSegmented CarsSegmentation TreesHow does it works?How does it works?Inexact Matching: Structural NoiseUse tree edit distance insteadTree Edit Distance• Editor Operations : costs ~ Dissimilarity(x,y)– Remove a node– Add a node – Replace a node-+rMetaphor: String Edit Distance• Unifying Editor Operations– Remove a node– Add a node Æ (removal on partner) – Replace a node Æ (paired removal on both string/tree)AABBBBCCEdit : Add YAABBYBBCCEdit : Remove YAABBXBBCCEdit : Replace X with YAABBYBBCCEdit : Remove XEdit : Remove YTree Edit Distance• Editor Operation (with costs)– Two way removal onlytt’u=E1(t)∩E2(t’)Dist.(t, t’) = Dist.(t, u) + Dist(u, t’)E():Sequenceof removalE1()E1()Reduce Edit-Distance matching to Non-edit matching• Transitive Closure• (see animation)Closure OriginalMatching CriteriaDivide and ConquerTry all pairs of (v, v’) combinations = O(|t| + |t’|)NP-complete Æ QP approx. O(|Cvv’|)Segment out all the cars….fused tree model for carsUnseen imageTraining imagesSegmented CarsSegmentation TreesModel Model GenerationGenerationModel: Union of Subtrees…NP-Hard1.Pairwise matching2.One by one unionSub-optimalOptimal=∪Next TreeT = T u TnextCategory ModelSegment out all the cars….fused tree model for carsUnseen imageTraining imagesSegmented CarsSegmentation TreesTesting:Testing:SegmentationSegmentationSegment out all the carsfused tree model for carsUnseen imageSegmented CarsTesting: Detect & SegmentationMaximal CommonMaximal CommonSubtreeSubtreeMatching Matching Match : (Similarity > Thresh) Æ(precision/recall)Performace EvaluatonResults (Caltech 101 Face)Varying Matching Thresh. Æ(precision/recall)Results (UIUC Car Side View)#positive/#training: 5/10 vs 10/20(2hr on P4-2.4G/2G)Results (Caltech 101 Face)#positive/#training: 3/6 vs 6/12Rotation InvariantCaltech (Cars Rear View)#positive/#training: 10/20Conclusion• Contribution– Good Image Representation Æ Seg. Tree• Small amount of training data– Cf. Statistical Learning/Clustering• Ex. Visual Words + pLSA• Allow Non-category Images noise• Allow occlusion (disconnected regions)Region DescriptorAnnotated Recursive TreeSalient FlowSliced area histogramRegion AreaGraylevelxxxxxxxxTopologicalGeometricPhotmetricThank you• QuicktopicCf. Visual Words+pLSA• Visual Words recognize connected object only• Tree Matching is more conservative due to intersectionCf. Visual Words +pLSAVisual Words/pLSATree matchingCaltech FacesVisual Words/pLSATree matchingReSPEC(Use Color