C280 Computer VisionC280, Computer VisionProf. Trevor [email protected] 14: Discriminative Kernels for RecognitionTwo Lectures agoTwo Lectures ago…•Scanning window paradigmScanning window paradigm•GISTHOG•HOG• Boosted Face Detection• Local-feature Alignment; from Roberts to Lowe...• BOW IndexingLast Lecture: Topic Models for RecognitionGuest lecture by Kate Saenko:• Dataset issues • Topic models for category discovery [Sivic05]• Category discovery from web [Fergus05]• Bootstrapping a category model [Li07]•Using text in addition to image [Berg06]gg[g]• Learning objects from a dictionary [Saenko08]Last Lecture SummaryLast Lecture Summary• The web contains unlimited, but extremely noisy object tdtcategory data• The text surrounding the image on the web page is an gg pgimportant recognition cue•Topic models (pLSA, LDA, HDP, etc.) are useful forTopic models (pLSA, LDA, HDP, etc.) are useful for discovering objects in images and object senses in text•Bootstrap model from small amount of labeled or weaklyBootstrap model from small amount of labeled or weakly labeled data•Still an open research problem!•Still an open research problem!Today:Discriminative KernelsToday: Discriminative Kernels•SVM-BOWSVMBOW• Pyramid and Spatial-Pyramid matchFtIt ti K l•Fast Intersection Kernels• Latent-part SVM modelsObject categorization: Object categorization: the statistical viewpointthe statistical viewpointthe statistical viewpointthe statistical viewpoint)|(imagezebrap)|(gp)(ezebra|imagnopvs.)(ezebra|imagnop•Bayes rule:•Bayes rule:)()|()|( zebrapzebraimagepimagezebrap⋅=)()|()|( zebranopzebranoimagepimagezebranopposterior ratiolikelihood ratio prior ratioFei-Fei, Fergus, Torralba, CVPR 2007 SCObject categorization: Object categorization: the statistical viewpointthe statistical viewpointthe statistical viewpointthe statistical viewpoint)()|()|(zebrapzebraimagepimagezebrap)()()|()|()|()|(zebranopzebrapzebranoimagepzebraimagepimagezebranopimagezebrap⋅=posterior ratiolikelihood ratio prior ratio• Discriminative methods model posterior• Generative methods model likelihood and priorpriorFei-Fei, Fergus, Torralba, CVPR 2007 SCDiscriminative• Direct modeling of )|()|(imagezebranopimagezebrapZebraDecision)|(imagezebranopZebraNon-zebraecsoboundaryFei-Fei, Fergus, Torralba, CVPR 2007 SCGenerative• Model and )|( zebraimagep) |( zebranoimagep)|( zebranoimagep)|( zebraimagepLow MiddleHigh MiddleÆLow1.Feature detection 1.Feature detection and representationand representation• Regular grid– Vogel & Schiele, 2003– Fei-Fei & Perona, 2005It t itdt t•Interest point detector– Csurka, Bray, Dance & Fan, 2004–Fei-Fei & Perona 2005FeiFei & Perona, 2005– Sivic, Russell, Efros, Freeman & Zisserman, 2005• Other methods– Random sampling (Vidal-Naquet & Ullman, 2002)– Segmentation based patches (Barnard, Duygulu, Forsyth de Freitas Blei Jordan 2003)Forsyth, de Freitas, Blei, Jordan, 2003)Fei-Fei, Fergus, Torralba, CVPR 2007 SC1.Feature 1.Feature detectiondetection and and representationrepresentationNormalize Compute SIFToaepatchDetect patchesSIFT descriptor[Lowe’99]Detect patches[Mikojaczyk and Schmid ’02][Mata, Chum, Urban & Pajdla, ’02] [Sivic & Zisserman’03][Sivic & Zisserman, 03]Slide credit: Josef Sivic1.Feature 1.Feature detectiondetection and and representationrepresentation…Fei-Fei, Fergus, Torralba, CVPR 2007 SC2. Codewords dictionary formation2. Codewords dictionary formation…Fei-Fei, Fergus, Torralba, CVPR 2007 SC2. Codewords dictionary formation2. Codewords dictionary formation…Vector quantizationSlide credit: Josef Sivic2. Codewords dictionary formation2. Codewords dictionary formationFei-Fei et al. 2005Image patch examples of codewordsImage patch examples of codewordsSivic et al. 20053. Image representation3. Image representationuencyfrequ…..codewordsFei-Fei, Fergus, Torralba, CVPR 2007 SCRepresentationRepresentation2.2.feature detection& representationcodewords dictionarycodewords dictionary1.1.image representation333.3.Fei-Fei, Fergus, Torralba, CVPR 2007 SCLearning and RecognitionLearning and Recognitioncodewords dictionarycodewords dictionarycategorycategorycategory modelscategory modelscategorycategorydecisiondecisioncategory modelscategory models(and/or) classifiers(and/or) classifiersLearning and RecognitionLearning and Recognition1Generative method:345itiesp(x|C2)1.Generative method: - graphical models0 0.2 0.4 0.6 0.8 10123class densitiep(x|C1)x2Discriminative method:0.81abilitiesp(C1|x) p(C2|x)2.Discriminative method: - SVM0 0.2 0.4 0.6 0.8 100.20.40.60.8posterior probabxcategory modelscategory modelscategory modelscategory models(and/or) classifiers(and/or) classifiersFei-Fei, Fergus, Torralba, CVPR 2007 SCDiscriminative methods based on ‘b f d ’ i‘bag of words’ representationZebraDecisioneb aNon-zebraboundaryFei-Fei, Fergus, Torralba, CVPR 2007 SCDance et alDance et al.……Detect or sample Describe Quantize to form …Detect or sample featuresDescribe featuresList of positions, Associated list of Qua t e to o bag of words vector for the imagescales, orientationsd-dimensional descriptorsSVM23Chris Dance, Jutta Willamowski, Lixin Fan, Cedric Bray, and Gabriela Csurka. Visual categorization with bags of keypoints. In ECCV International Workshop on Statistical Learning in Computer Vision, 2004.EmbeddingEmbedding Learning with Kernels, Scholkopf and Smola, 2002KernelsKernels•linear classifier:linear classifier:• Kernel classifier:[Dance et al.]Learning with Kernels, Scholkopf and Smola, 2002Tried linear, quadratic, cubic; linear had best performance….K( , ) = K( , ) = < , >[Dance et al.]SVM:Naïve Bayes:[Dance et al.]How to Compare Sets of Features?• Each instance is unordered set of vectors• Varying number of vectors per instance??MIT CSAILVision interfacesRecap: Conventional Approaches“Voting” – for each patch, find the most similar patch in database, and vote for the image containing that th[S h id L T t l t l ]patch.[Schmid, Lowe, Tuytelaars et al.]+12345150…MIT CSAILVision interfacesRecap: Conventional Approaches“Voting” – for each patch, find the most similar patch in database, and vote for the image containing that th[S h id L T t l t l ]patch.[Schmid, Lowe, Tuytelaars et al.]2345150…“Bag of Words” – quantize descriptor space, represent each image as a histogram over prototypes; use