Image Retrieval: Current Techniques, Promising Directions, and Open IssuesPresentation OutlineHistory of Image RetrievalLimitations of text-based approachOutlineWhat is CBIR?CBIR – A sample search querySample QuerySample CBIR architectureSlide 10Feature ExtractionMean ColorHistogramColor LayoutExample: Color layoutTextureSegmentation issuesSlide 18Problem of high dimensions2-dimensional space3-dimensional spaceNow, imagine…Slide 23IBM’s QBICQBIC – Search by colorQBIC – Search by shapeQBIC – Query by sketchVirageVisualSEEkISearch – my own systemSlide 31Slide 32Feature selection in ISearchDatabase Admin facility in ISearchSlide 35Open issuesSlide 37ConclusionAcknowledgementsReferencesSlide 41Image Retrieval: Current Image Retrieval: Current Techniques, Promising Techniques, Promising Directions, and Open IssuesDirections, and Open IssuesYong Rui, Thomas Huang and Shih-Fu Yong Rui, Thomas Huang and Shih-Fu ChangChangPublished in the Journal of Visual Published in the Journal of Visual Communication and Image Communication and Image Representation.Representation.Presented by: Deepak BotePresentation OutlinePresentation OutlineHistory of image retrieval – Issues facedHistory of image retrieval – Issues facedSolution – Content-based image retrievalSolution – Content-based image retrievalFeature extractionFeature extractionMultidimensional indexingMultidimensional indexingCurrent SystemsCurrent SystemsOpen issuesOpen issuesConclusionConclusionHistory of Image History of Image RetrievalRetrievalTraditional text-based image search Traditional text-based image search enginesenginesManual annotation of imagesManual annotation of imagesUse text-based retrieval methodsUse text-based retrieval methodsE.g. E.g. Water liliesFlowers in a pond <Its biological name>Limitations of text-based Limitations of text-based approachapproachProblem of image annotationProblem of image annotationLarge volumes of databasesLarge volumes of databasesValid only for one language – with image Valid only for one language – with image retrieval this limitation should not existretrieval this limitation should not existProblem of human perceptionProblem of human perceptionSubjectivity of human perceptionSubjectivity of human perceptionToo much responsibility on the end-userToo much responsibility on the end-userProblem of deeper (abstract) needsProblem of deeper (abstract) needsQueries that cannot be described at all, but Queries that cannot be described at all, but tap into the visual features of images.tap into the visual features of images.OutlineOutlineHistory of image retrieval – Issues facedHistory of image retrieval – Issues facedSolution – Content-based image retrievalSolution – Content-based image retrievalFeature extractionFeature extractionMultidimensional indexingMultidimensional indexingCurrent SystemsCurrent SystemsOpen issuesOpen issuesConclusionConclusionWhat is CBIR?What is CBIR?Images have rich content.Images have rich content.This content can be extracted as This content can be extracted as various content features:various content features:Mean color , Color Histogram etc…Mean color , Color Histogram etc…Take the responsibility of forming Take the responsibility of forming the query away from the user.the query away from the user.Each image will now be described by Each image will now be described by its own features.its own features.CBIR – A sample search CBIR – A sample search queryqueryUser wants to search for, say, many rose User wants to search for, say, many rose imagesimagesHe submits an existing rose picture as query.He submits an existing rose picture as query.He submits his own sketch of rose as query.He submits his own sketch of rose as query.The system will extract image features for The system will extract image features for this query.this query.It will compare these features with that of It will compare these features with that of other images in a database.other images in a database.Relevant results will be displayed to the Relevant results will be displayed to the user.user.Sample QuerySample QuerySample CBIR Sample CBIR architecturearchitectureOutlineOutlineHistory of image retrieval – Issues facedHistory of image retrieval – Issues facedSolution – Content-based image retrievalSolution – Content-based image retrievalFeature extractionFeature extractionMultidimensional indexingMultidimensional indexingCurrent SystemsCurrent SystemsOpen issuesOpen issuesConclusionConclusionFeature ExtractionFeature ExtractionWhat are image features?What are image features?Primitive featuresPrimitive featuresMean color (RGB)Mean color (RGB)Color HistogramColor HistogramSemantic featuresSemantic featuresColor Layout, texture etc…Color Layout, texture etc…Domain specific featuresDomain specific features Face recognition, fingerprint matching Face recognition, fingerprint matching etc…etc…General featuresMean ColorMean ColorPixel Color Information: R, G, BPixel Color Information: R, G, BMean component (R,G or B)= Mean component (R,G or B)= Sum of that component for all pixels Sum of that component for all pixels Number of pixelsNumber of pixels PixelHistogramHistogramFrequency count of each individual Frequency count of each individual colorcolorMost commonly used color feature Most commonly used color feature representationrepresentation Image Corresponding histogramColor LayoutColor LayoutNeed for Color LayoutNeed for Color Layout Global color features give too many false Global color features give too many false positivespositivesHow it works:How it works: Divide whole image into sub-blocksDivide whole image into sub-blocks Extract features from each sub-blockExtract features from each sub-blockCan we go one step further?Can we go one step further?Divide into regions based on color feature Divide into regions based on color feature concentrationconcentrationThis process is called segmentation.This process is called segmentation.Example: Color layoutExample: Color layout** Image adapted from Smith and Chang : Single Color Extraction and Image QueryTextureTextureTexture – innate property of all surfacesTexture – innate property of all surfacesClouds, trees, bricks, hair etc…Clouds, trees, bricks, hair etc…Refers to visual
View Full Document
Unlocking...