ECE160 / CMPS182 MultimediaNational Research PrioritiesMultimedia RecognitionWide-area SurveillanceSlide 5Slide 6How to Organize these Photos?Image Organization & RetrievalEXTENTTM = contEXT + contENTEXTENTTMAugmented ImagesContext from Space/TimeExample of Using Three Pieces of InformationMaui Sunsets can be obtained from Space/TimeUse content for verificationUse content to transfer metadataSummarize of the exampleAre They Similar?Slide 19Are They Similar?Conveying PerceptionSlide 22Keyword RetrievalSlide 24Slide 25Slide 26Slide 27Recogintion of ContentRecognitionSlide 30Slide 31Web 1.0 vs. Web 2.0Web 2.0FotofitiECE160 Spring 2009 Lecture 20 1ECE160 / CMPS182MultimediaLecture 20: Spring 2009Image Recognition and RetrievalECE160Spring 2009 Image Recognition 2National Research PrioritiesEnergy TechnologiesFuel efficient enginesReplacement energy to fossil fuelsLighter, longer-duration batteriesBioengineering/BioinformaticsGenes  diseaseDisease  medicineSearch with Multimedia Content Video surveillance Photo interpretationECE160Spring 2009 Image Recognition 3Multimedia RecognitionVideo surveillancePhoto interpretationECE160Spring 2009 Image Recognition 4Wide-area Surveillanceadvertisement of objectvideo.comECE160Spring 2009 Image Recognition 5Surveillance Scenarios(1) Intrusion Detection(1) Intrusion Detection (2) Passenger Screening(2) Passenger Screening(3) Perimeter Monitoring(3) Perimeter MonitoringZZUse biometric facial recognition to identify individuals of interest through existing closed circuit TV surveillanceMonitor and alert on tailgating, loitering, exit/closed entry, other unauthorized accessObject tracking and biometric facial recognition to determine vehicles and humans exhibiting suspicious behavior(4) Unattended Baggage(4) Unattended BaggageCopyright © 2004 Proximex Corp.Copyright © 2004 Proximex Corp.Identify unattended baggage (or other objects) left for long periods of timeECE160Spring 2009 Image Recognition 6Multimedia RecognitionVideo surveillancePhoto interpretationECE160Spring 2009 Image Recognition 7How to Organize these Photos?ECE160Spring 2009 Image Recognition 8Keyword-based Manual labeling is subjective, cumbersomeThe aliasing problemContent-basedPromising for general semantics: outdoor, landscape, flowers, people, etc. Not enough for w h-queries (where, who, when, or what)Image Organization & RetrievalECE160Spring 2009 Image Recognition 9EXTENTTM = contEXT + contENTContext Spatial (location)Temporal Social OthersContent Perceptual features, such as color, texture, and shapeHolistic features and local featuresECE160Spring 2009 Image Recognition 10EXTENTTMContentSpatial TemporalSocial OthersECE160Spring 2009 Image Recognition 11Augmented Images+Cameraphones with high-quality lens can record location, time, camera parameters, and voice=ECE160Spring 2009 Image Recognition 12Context from Space/TimeGPS or CellID data Into place names Time-based groupingInto meaningful “events”From place names and timeTime of dayWeatherECE160Spring 2009 Image Recognition 13Example of Using Three Pieces of Information ContentSpatial TemporalECE160Spring 2009 Image Recognition 14Maui Sunsetscan be obtained from Space/TimeECE160Spring 2009 Image Recognition 15Use content for verificationECE160Spring 2009 Image Recognition 16Use content to transfer metadataECE160Spring 2009 Image Recognition 17Summarize of the exampleDerived from ContextDerive time of the dayObtain weatherVerify contentUse of ContentVerify contextTransfer contextMuch more…ECE160Spring 2009 Image Recognition 18Are They Similar?ECE160Spring 2009 Image Recognition 19Are They Similar?ECE160Spring 2009 Image Recognition 20Are They Similar? In terms of what?What is the user’s perception?ECE160Spring 2009 Image Recognition 21Conveying Perception Image DatabasesConveyed via ExamplesUse a sunset picture (or pictures) to find more sunset imagesWhere does the perfect example come from?ECE160Spring 2009 Image Recognition 22Conveying Perception Internet SearchesConveyed via KeywordsECE160Spring 2009 Image Recognition 23Keyword Retrieval ProsA user-friendly paradigmConsAnnotation is a laborious processAnnotation quality can be subparAnnotation can be subjectiveSynonymsECE160Spring 2009 Image Recognition 24Conveying Perception Image DatabasesConveyed via ExamplesUse a sunset picture (or pictures) to find more sunset imagesWhere does the perfect example come from?ECE160Spring 2009 Image Recognition 25Are They Similar?ECE160Spring 2009 Image Recognition 26Are They Similar?ECE160Spring 2009 Image Recognition 27Are They Similar? In terms of what?What is the user’s perception?ECE160Spring 2009 Image Recognition 28Recogintion of ContentECE160Spring 2009 Image Recognition 29RecognitionECE160Spring 2009 Image Recognition 30RecognitionECE160Spring 2009 Image Recognition 31clouds vs. wavesECE160Spring 2009 Image Recognition 32Web 1.0 vs. Web 2.0UserUser UserContentContent (Image/Video)ContentContentContentContentContentContent (text)ContentContentContentContentContentUserUserUser UserECE160Spring 2009 Image Recognition 33Web 2.0 Content + Users + InteractionsCollect rich, organized contentAttract users & interactions To provide metadataTo provide new contentImprove search qualityWith new metadata and dataVia social-network structureECE160Spring 2009 Image Recognition 34User managementUser managementPhoto uploadingSingle/multipleUpload wizardPhoto uploadingBuilding social networkSocial networksEvent managementEvent managementPhoto searchPhoto searchMetadata collectionMetadata fusionImage annotationMetadata collection - contextual - content Metadata fusion Annotate photos External functionalitiesInternal