Intelligent Image Database Searching systemJelena TešicAdvisor: B.S. ManjunathECE 277A project presentationProject Goaln Bridge between n Categorical Image Search Engine(Sumengen, Newsam – Vision Research Lab @ UCSB)n Intelligent Image-Database Search Engine(Vision Group @ University of Bristol, UK)n Improve an image query system, based on user’s responsen Use of user’s feedback for Neural Network Trainingn Queries are iteratively refinedSemantics Based WWW Image Search Enginen Semantic classification of imagesn Categorical indexing of images and associated text.n Combining text and image primitives, such as color and texture for efficient search and retrievaln Using relevance feedback to help users find what they are searching for.n http://maya.ece.ucsb.edu/cgi-bin/imsearch/main.cgiResearch Directionsn Database indexing n combination with existing indexing methodsn clustering (multiple features) n Better retrieval results n User’s feedbackn Incorporate a perceptual dissimilarity measure information into access building processn Preprocessing step increases overall performancen Data statisticsn Keep it fairly simple and scalableIntelligent Image-Database Searching n Computer vision system for large WWW databasesn User defines objects of interest and similar objects in the database are then found automatically and returned as a set of thumbnails. n The user then selects which of these match his search n This allows refinement of the search as further positive and negative examples of the object of interest.Feature-based search n Hard to determine a set of useful features and their relationship(local & global features)n Relevant feature selection is automaticn Exhaustively seek for useful subsets is computationally expensiven Single class classification system that its progressively ‘tuned’ in its ability to identify the class of relevant objectsImplementationClassification using NNn Multi-Layer Perceptron networksn Requires large set of classified training data for acceptable resultsn Single feature searchn Radial Basis Function networks (Michelli, 1986.)n trained in a similar mannern simpler mathematical interpretation n the initial parameters to be determined using clustering algorithms. n preliminary classification of the data before sufficient examples are available to perform full optimization n refine the feature set as more examples are identified.Radial Basis Functionsn Radial basis function (RBF) networks have a static Gaussian function as the nonlinearity for the hidden layer processing elements. n The Gaussian function responds only to a small region of the input space where the Gaussian is centered. n The key to a successful implementation of these networks is to find suitable centers for theGaussian functions. This can be done with supervised learning, but an unsupervised approach usually produces better results.Radial Basis Functionsn Unsupervised layer: n The Gaussian centers and the widths are derived from the input data. n The centers are encoded within the weights of the unsupervised layer using competitive learning. n During the unsupervised learning, the widths of theGaussians are computed based on the centers of their neighbors. n The output of this layer is derived from the input data weighted by a Gaussian mixture.Radial Basis Functionn RBF network finds the input to output map using localapproximators. n Usually the supervised segment is simply a linear combination of the approximators. n Since linear combiners have few weights, these networks train extremely fast and require fewer training samples.n An RBF network is nonlinearif the basis function can move or change size or if there is more than one hidden layerData Preprocessingn The process can be described in three steps: preprocessing data, retrieving query images and off-line training of RBF networks. n Data preprocessing - a creation of a query database of image feature vectors. We use a database of 16 000 feature vectors. Shawn Newsam and Baris Sumengen originally developed the database for the Categorical image search engine. The feature vector is combination of a 24 dimensional texture feature vector and a 64 dimensional color feature vectors.Queryn Query is initiated by the image, region or feature of interest from a key image.n System selects a preliminary set of mages that may contain the desired feature n The user is required to state, for each of the retrieved images, if it matches the original image or not. n The output corresponds to user’s feedbackOff line trainingn Complex training algorithm to create a single-class classifier. n A set of Radial Basis Function (RBF) nodes is placed appropriately in the feature space amongst the training data vectors. n An RBF node is placed as a center of each cluster in the feature space.n The outputs of the nodes are connected to a single-layer two output perceptron.Refinement Processn The centroids of RBF are bootstrapped using the LVQ algorithm. n The number of RBF nodes created depends upon the output of the LVQ pass. n A two-layer MLP is used to correlate the activations of RBF nodes to the input feature data n The multi-stage training process – centroids, output layer , distribution then output layer again – continues until parameter movement is minimized.Implementationn MATLAB RBF functions n Mark Orr, Edinburgh University n Regression treesn First step:n A test set of images is from one semantic categoryn Comparison with Categorical Image Searchn System improvementn Initial experimentsImplementationInitial ImplementationThree passes through sunglasses categoryConclusionn Report and more examples:http://vision.ece.ucsb.edu/~jelena/research/n The goal of this project was to explore if Neural Networks can efficiently capture the relationship among features for better image retrievaln We need to conduct far more experiments on a larger scale in order to determine the use of such a systemAcknowledgementn Kris BruvoldProgramming, debugging and implementation n Prof.ManjunathFurther Research n http://www-iplab.ece.ucsb.edu/demos.htmn http://www.cs.bris.ac.uk/Research/Vision/search.htmln