DOC PREVIEW
Berkeley COMPSCI 294 - Learning Object Categories from contaminated data

This preview shows page 1-2-3-4-24-25-26-50-51-52-53 out of 53 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 53 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Learning Object Categories from contaminated dataSlide 2Slide 3Internet Search EnginesSlide 5Slide 6Slide 7Slide 8Slide 9Specification of learning problemSlide 11Slide 12Slide 13Visual wordsSlide 15Slide 16Slide 17Proposing bounding boxesComparison between pLSA modelsTraining pLSA models from Google imagesGoogle’s variable search performancePicking the best topicOverall learning schemeMotorbike – pLSAMotorbike – TSI-pLSACar Rear – TSI-pLSASlide 30Slide 31Slide 32Slide 33Slide 34Constellation model Wrist watchSlide 36Slide 37Slide 38Slide 39Slide 40Watch modelMotorbike modelAirplane modelConfusion tableRetrieval ExperimentComparison to other methodsConclusionsPASCAL cars - classificationPASCAL Motorbikes - classificationGoogle search applicationPASCAL Cars - localizationPASCAL Motorbikes - localizationPASCAL Cars localization examplesPASCAL Motorbikes localization examplesSlide 70Slide 71Learning Object Categories from contaminated dataRob FergusLi Fei-FeiPietro PeronaAndrew ZissermanOxford UniversityCalifornia Institute of TechnologyContaminated learning environmentsHome photo collectionsMobile robotInternet Search Engines• Automatically obtain 100’s of images given just keyword• If we can learn from this data, we can get models of any objectGround TruthGood imageJunk imageIntermediate imagesStatistics of Google ImagesSpecification of learning problem•No labels anywhere!•Different aspects•No segmentation•Clutter•Contaminated dataset•Lots of images (~500)•Two different models–pLSA-based models–Constellation model•Two objectives–Improving Google’s Image Search–Learn models for use in open-world setting•Evaluate on standard datasetsStructure of talk1. pLSA-based methods• Introduced by Hofmann in text analysis field• Latent Dirichlet Allocation (LDA) – Blei and Jordan• Adapted to visual data by: - Sivic, Russell et al. (Unsupervised object category discovery) ICCV ‘05 - Fei-Fei and Perona (Scene analysis using LDA) CVPR ‘05Text domain Image domaind Document Imagez Topic Objectw WordVQ’d appearance of regionProbabilistic Latent Semantic Analysis (pLSA)• Need to choose # topics (Z)Visual words•4 types of region detector–Kadir & Brady–Multi-scale Harris–Difference of Gaussians–Edge based•SIFT representation –Histograms of orientations (72 dimensions)•VQ using pre-computed codebook of size 350.•Color not used1. Improvements on pLSA: ABS-pLSA•No spatial information used in pLSA•Simplest form of spatial model:•Joint spatial/word model–Quantize location of region within image–Absolute coordinate frame2. Improvements on pLSA: TSI-pLSA•ABS-pLSA uses absolute coordinate frame–Cannot handle translation or scaling•Introduce sub-window conditioned on hidden variable c:• c is a 4-d vector – gives bounding box of object• Gives (T)ranslation and (S)cale (I)nvariance.Proposing bounding boxes• Use pLSA to propose bounding boxes in a bottom-up manner• Use regions weighted by P(w|z). • Fit Gaussian mixture model with (C=1 & C= 2) components for each topic:C = 1 componentC = 2 components• Gives us a set of possible bounding boxesComparison between pLSA modelsPlain pLSA ABS – pLSA TSI-pLSAPASCAL Cars31.7 30.8 25.8PASCAL Motorbikes33.7 30.2 25.7• PASCAL object recognition challenge datasets• Classification taskTraining pLSA models from Google images•Multiple topics can handle polluted data–Each topic models a visually consistent component of data•Different aspects handled by–Different topics–Multimodal nature of densities•TSI-pLSA can handle translation and scaling of object within image•Everything automatic except for:1. Number of topics to use (Z)? Fix Z =82. How to pick topic belonging to good images?Google’s variable search performancePicking the best topic•Use Google’s automatic translation tool to translate keyword•Use: German, French, Italian, Spanish, Chinese, English•Take first 5 images returned using translated keywords to give validation setOverall learning schemeCollect images from GoogleKeywordFind regionsVQ regionsLearn 8 topic pLSA modelPropose bounding boxesLearn 8 topic TSI-pLSA modelTranslate keywordCollect validation setPick best topicClassifierMotorbike – pLSAMotorbike – TSI-pLSACar Rear – TSI-pLSA2. Constellation ModelModel: Constellation of PartsFischler & Elschlager 1973Yuille ‘91Brunelli & Poggio ‘93Lades, v.d. Malsburg et al. ‘93Cootes, Lanitis, Taylor et al. ‘95Amit & Geman ‘95, ‘99 Perona et al. ‘95, ‘96, ’98, ’00Agarwal & Roth ‘02Main issues:• measuring the similarity of parts• representing the configuration of partsDetected regionsMotorbikesSamples from appearance model•Background model explains regions not allocated to model partsApplication of Constellation Model to Google data•3 types of region detector–Kadir & Brady–Multi-scale Harris–Curves•Learn set of models, each with different combinations of feature types•Pick best combination using validation set•Intuition: foreground model picks out good images while background model explains junkConstellation model Wrist watch•Loose shape model•High variance •K = Kadir & Brady; C = CurveConstellation model Face•K = Kadir & Brady; H = multi-scale Harris; C = Curve• High variance• Picks out hairlineImproving Google’s Image SearchRecall-Precision curvesComparison between methodsApplication of model to standard datasetsWatch modelMotorbike modelAirplane modelConfusion tableRetrieval Experiment• Dataset of 2148 images from all 7 classesComparison to other methodsDatasetTSI-pLSA CM CMpLSA (Sivic, Russel et al)Opelt et al. Leibe et al.Supervision None None Img. labels None Img. Labels SegmentedAirplane 15.5 21.0 6.3 3.4 7.8 -Cars Rear 16.0 31.8 2.3 21.4 8.9 6.1Face 20.7 48.8 8.3 5.3 6.5 -Leopard 13.0 23.0 11.0 - - -Motorbike 6.2 25.8 3.3 15.4 7.8 6.0• Error rates at point of equal error on ROC curveConclusions•Black box methods – only keyword needed, for learning object categories, training directly from Google images•Extensions to pLSA to incorporate spatial information•Shown how Google’s Image search can be improved•pLSA-based methods can produce models for use in open-world scenarios, unlike the Constellation Model. •Potential use as category-level priors for other applications.PASCAL cars - classification0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9


View Full Document

Berkeley COMPSCI 294 - Learning Object Categories from contaminated data

Documents in this Course
"Woo" MAC

"Woo" MAC

11 pages

Pangaea

Pangaea

14 pages

Load more
Download Learning Object Categories from contaminated data
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Learning Object Categories from contaminated data and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Learning Object Categories from contaminated data 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?