Animals on the WebSlide 2Slide 3Slide 4Slide 5Slide 6Flickr Search - monkeyOur ResultsSlide 9Museum and Library CollectionsWeb CollectionsVideoConsumer ProductsGeneral ApproachPrevious Work - Words & PicturesSlide 16Slide 17Slide 18Slide 19Slide 20Slide 21Select ExemplarsSlide 23Slide 24Slide 25Slide 26Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33SummaryAcknowledgementsUCBComputer VisionAnimals on the WebTamara L. BergSUNY Stony BrookUCBComputer Vision I want to find lots of good pictures of monkeys… What can I do?UCBComputer VisionGoogle Image Search -- monkeyUCBComputer VisionGoogle Image Search -- monkeyUCBComputer VisionGoogle Image Search -- monkeyUCBComputer VisionGoogle Image Search -- monkeyWords alone won’t workUCBComputer VisionFlickr Search - monkey Even with humans doing the labeling, the data is extremely noisy -- context, polysemy, photo sets Words alone still won’t work!UCBComputer VisionOur ResultsUCBComputer Vision Consumer Photo CollectionsOver the hills and far awayRoad, Hills, Germany, Hoffenheim, Outstanding Shots, specland, Baden-WuerttembergHeavenlyPeacock, AlbinoPeacock, WhiteBeauty, Birds, Wildlife, FeathredaleWildlifePark, PictureAustralia, ImpressedBeautyEnd of the world - Verdens Ende - The lighthouse 1Verdens ende, end of the world, norway, lighthouse, ABigFave, vippefyr, wood, coalFlickr – 3 billion photographs, several million uploaded per dayUCBComputer VisionMuseum and Library Collections Fine Arts Museum of San Francisco (82,000 images)Woman of Head Howard H G Mrs Gift America North bust States United Sculpture marblebowl stemmed small Irridescent glass New York Public Library Digital CollectionThe new board walk, Rockaway, Long IslandPart of New England, New York, east New Iarsey and Long Iland.UCBComputer VisionWeb CollectionsBillions of Web PagesUCBComputer VisionVideoOUTSIDE IN THE RAIN THE SENATOR WEARING HIS UH BASEBALL CAP A BOSTON RED SOX CAP AS HE TALKED TO HIS SUPPORTERS HERE IN THE RAIN THE UH SENATOR THEY'RE DOING HIS BEST TO TRY TO MAKE HIS CASE THAT HE WILL BE THE MAN FOR THE MIDDLE CLASS AND UH TRY TO CONVINCE HIS SUPPORTERS TO EXPRESS THEIR SUPPORT THROUGH A VOTE ON TUESDAY IN THERE WE ARE TWENTY FOUR HOURS FROM THE GREAT MOMENT THAT THE WORLD IN AMERICA IS WAITING FOR IT I NEED TO YOU IN THESE HOURS TO GO OUT AND DO THE HARD WORK NOT ON THOSE DOORS MAKE THOSE PHONE CALLS TO TALK TO FRIENDS TAKE PEOPLE TO THE POLLS HELP US CHANGE THE DIRECTION OF THIS GREAT NATION FOR THE BETTER CAN YOU IMAGINE A UH SENATOR BEGINNING HIS DAY IN FLORIDA TODAYTrecVid 2006UCBComputer VisionConsumer ProductsMarc by Marc JacobsAdorable peep-toe pumps, great for any occasion. Available in an array of uppers. Metallic fabric trim and bow detail. Metallic leather lined footbed. Lined printed design.Leather sole. 3 3/4" heel.Zappos.comsoft and glassy patent calfskin trimmed with natural vachetta cowhide, open top satchel for daytime and weekends, interior double slide pockets and zip pocket, seersucker stripe cotton twill lining, kate spade leather license plate logo, imported2.8" drop length14"h x 14.2"w x 6.9"dKatespade.comIt's the perfect party dress. With distinctly feminine details such as a wide sash bow around an empire waist and a deep scoopneck, this linen dress will keep you comfortable and feeling elegant all evening long. * Measures 38" from center back, hits at the knee. * Scoopneck, full skirt. * Hidden side zip, fully lined. * 100% Linen. Dry clean.bananarepublic.comE-commerce transactions in 2004, 2005, 2006 of $145 billion, $168 billion, and $198 billion (Forrester Research).UCBComputer VisionGeneral Approach- Vision alone won’t solve the problem. - Text alone won’t solve the problem. -> Combine the two!UCBComputer VisionPrevious Work - Words & PicturesLi and Wang, PAMI 2003Barnard et al,CVPR 2001Clustering ArtAuto-AnnotationBarnard et al, JMLR 2003Labeling RegionsYanai et al, MIR 2005Image ClassificationUCBComputer VisionAnimals on the WebExtremely challenging visual categories.Free text on web pages.Take advantage of language advances.Combine multiple visual and textual cues.UCBComputer VisionWords alone won’t workImage SearchUCBComputer VisionGoal:Classify images depicting semantic categories of animals in a wide range of aspects, configurations and appearances. Images typically portray multiple species that differ in appearance.UCBComputer VisionAnimals on the Web Outline:Harvest pictures of animals from the web using Google Text Search.Select visual exemplars using text based information.Use visual and textual cues to extend to similar images.UCBComputer VisionHarvested Pictures14,051 images for 10 animal categories.12,886 additional images for monkey category using related monkey queries (primate, species, old world, science…)UCBComputer VisionText Model Latent Dirichlet Allocation (LDA) on the words in collected web pages to discover 10 latent topics for each category. Each topic defines a distribution over words. Select the 50 most likely words for each topic.1.) frog frogs water tree toad leopard green southern music king irish eggs folk princess river ball range eyes game species legs golden bullfrog session head spring book deep spotted de am free mouse information round poison yellow upon collection nature paper pond re lived center talk buy arrow common princeExample Frog Topics:2.) frog information january links common red transparent music king water hop tree pictures pond green people available book call press toad funny pottery toads section eggs bullet photo nature march movies commercial november re clear eyed survey link news boston list frogs bull sites butterfly court legs type dot blueUCBComputer VisionSelect ExemplarsRank images according to whether they have these likely words near the image in the associated page (word score)Select up to 30 images per topic as exemplars.2.) frog information january links common red transparent music king water hop tree pictures pond green people available book call press ...1.) frog frogs water tree toad leopard green southern music king irish eggs folk princess river ball range eyes game species legs golden bullfrog session head ...UCBComputer VisionSensesThere are multiple senses of a category within the Google search results. Ask the user to identify which of the 10 topics are relevant to their search. Merge.Optional second step of supervision – ask user to mark erroneously labeled exemplars.UCBComputer VisionImage Model Match Pictures of
View Full Document
Unlocking...