IntroductionPrevious Work in CBIRRecent Work in Shape-Based CBIRColor and Shape FeaturesColor Indexing ApproachShape RepresentationIndexing Scheme and QueryingColor IndexShape IndexQueryingSimilarity MeasureExperimental Results and PerformanceConclusionsColor and Shape Index for Region-Based ImageRetrievalB.G. Prasad1?, S.K. Gupta2, and K.K. Biswas21Department of Computer Science and Engineering,P.E.S.College of Engineering, Mandya, 571402, INDIA.2Department of Computer Science and Engineering,Indian Institute of Technology, New Delhi, 110016, INDIA.{bgprasad,skg,kkb}@cse.iitd.ernet.in,WWW home page: http://www.cse.iitd.ernet.in/˜ skgupta/Abstract. Most CBIR systems use low-level visual features for repre-sentation and retrieval of images. Generally such methods suffer fromthe problems of high-dimensionality leading to more computational timeand inefficient indexing and retrieval performance. This paper focuses ona low-dimensional color and shape based indexing technique for achiev-ing efficient and effective retrieval performance. We propose a combinedindex using color and shape features. A new shape similarity measureis proposed which is shown to be more effective. Images are indexed bydominant color regions and similar images form an image cluster storedin a hash structure. Each region within an image is further indexed bya region-based shape index. The shape index is invariant to translation,rotation and scaling. A JAVA based query engine supporting query-by-example is built to retrieve images by color and shape. The retrievalperformance is studied and compared with a region-based shape index-ing scheme.1 IntroductionThe past few years have seen many advanced techniques evolving in Content-Based Image Retrieval (CBIR). Applications like medicine, entertainment, edu-cation, manufacturing, etc. make use of vast amount of visual data in the formof images. This envisages the need for fast and effective retrieval mechanisms inan efficient manner. A major approach directed towards achieving this goal isthe use of low-level visual features of the image data to segment, index and re-trieve relevant images from the image database. Recent CBIR systems based onfeatures like color, shape, texture, spatial layout, object motion, etc., are cited in[1],[2]. Of all the visual features, color is the most dominant and distinguishingone in almost all applications.?This work is partly supported by the AICTE Young Teachers Career AwardC. Arcelli et al. (Eds.): IWVF4, LNCS 2059, pp. 716–725, 2001.c Springer-Verlag Berlin Heidelberg 2001Color and Shape Index for Region-Based Image Retrieval 7171.1 Previous Work in CBIRCurrent CBIR systems such as IBM’s QBIC [3],[4] allow automatic retrievalbased on simple characteristics and distribution of color, shape and texture.But they do not consider structural and spatial relationships and fail to cap-ture meaningful contents of the image in general. Also the object identificationis semi-automatic. The Chabot project [5] integrates a relational database withretrieval by color analysis. Textual meta-data along with color histograms formthe main features used. VisualSEEK [6] allows query by color and spatial layoutof color regions. Text based tools for annotating images and searching is pro-vided. A new image representation which uses the concept of localized coherentregions in color and texture space is presented by Carson et al. [7]. Segmentationbased on the above features called “Blobworld” is used and query is based onthese features.Some of the popular methods to characterize color information in images arecolor histograms [8],[9], color moments [10] and color correlograms [11]. Thoughall these methods provide good characterization of color, they have the problemof high-dimensionality. This leads to more computational time, inefficient index-ing and performance. To overcome these problems, use of SVD [9], dominantcolor regions approach [12],[13] and color clustering [14] have been proposed.1.2 Recent Work in Shape-Based CBIRShape also is an important feature for perceptual object recognition and classi-fication of images. It has been used in CBIR in conjunction with color and otherfeatures for indexing and retrieval.Shape description or representation is an important issue both in objectrecognition and classification. Many techniques, including chain code, polygo-nal approximations, curvature, fourier descriptors and moment descriptors havebeen proposed and used in various applications [15]. Recently, techniques usingshape measure as an important feature have been used for CBIR. Features suchas moment invariants and area of region have been used in [3],[16], but do notgive perceptual shape similarity. Cortelazzo [17] used chain codes for trademarkimage shap e description and string matching technique. The chain codes are notnormalized and string matching is not invariant to shape scale. Jain and Vailaya[18] proposed a shape representation based on the use of a histogram of edgedirections. But these are not scale normalized and computationally expensivein similarity measures. Mehrotra and Gary [19] used coordinates of significantpoints on the boundary as shape representation. It is not a compact represen-tation and the similarity measure is computationally exp ensive. Jagadish [20]proposed shape decomposition into a number of rectangles and two pairs of co-ordinates for each rectangle are used to represent the shape. It is not rotationinvariant.A region-based shape representation and indexing scheme that is translation,rotation and scale invariant is proposed by Lu and Sajjanhar [21]. It conforms tohuman similarity perception. They have compared it to Fourier descriptor model718 B.G. Prasad, S.K. Gupta, and K.K. Biswasand found their method to be better. But, the images database consists of only2D planar shapes and they have considered only binary images. Moreover, shapeswith similar eccentricity but different shapes are retrieved as matched images.Our aim is to extend this method to represent color image regions and augmentthe color index of our previous work [13] with the shape features. Our shapeindexing feature and similarity measure is different and shown to be effective inretrieval. A combined index based on color and shape has been implemented toimprove retrieval efficiency and effectiveness.The paper is organised as follows: Section 2 describes the color and shapefeatures used for indexing. The indexing scheme, querying and similarity