PageRank for Product Image Search Yushi Jing1 Shumeet Baluja2 1 College of Computing Georgia Institute of Technology Atlanta GA 2 Google Inc 1600 Amphitheater Parkway Mountain View CA WWW 2008 April 2008 Beijing China Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 1 45 Objective To incorporate visual aspects of images into selecting of images for describing products to buy Figure Google Product Search Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 2 45 Outline 1 Background Current Image Search Methods Applying Object Detection to Image Search Detecting and Applying Low Level Features 2 Approach and Algorithm Reranking Results From Image Search Computing Low Level Features From Features to Similarity From Similarity to Centrality From Centrality to Reranking 3 Experimental Results User Evaluation Subjective Click Study Objective Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 3 45 Outline 1 Background Current Image Search Methods Applying Object Detection to Image Search Detecting and Applying Low Level Features 2 Approach and Algorithm Reranking Results From Image Search Computing Low Level Features From Features to Similarity From Similarity to Centrality From Centrality to Reranking 3 Experimental Results User Evaluation Subjective Click Study Objective Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 4 45 Current Image Search Popular search engines for images do not use image features to rank images Instead the following are commonly used I I I anchor text image name surrounding text on the webpage Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 5 45 Reasons for Text Based Image Search Text based search is well studied and successful Object recognition still largely unsolved Computational complexity of computer vision tasks Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 6 45 Problem With Not Looking At Image Itself Results are often inconsistent and uncontrolled in terms of quality and in terms of content Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 7 45 Outline 1 Background Current Image Search Methods Applying Object Detection to Image Search Detecting and Applying Low Level Features 2 Approach and Algorithm Reranking Results From Image Search Computing Low Level Features From Features to Similarity From Similarity to Centrality From Centrality to Reranking 3 Experimental Results User Evaluation Subjective Click Study Objective Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 8 45 Previous Method to Use Vision to Rank Images The algorithm Construct models of categories of objects trained from top search results Rerank images based on their fit to the model The problem Assumption of one homogeneous object category per query is unrealistic Can potentially maximize relevance but not diversity Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 9 45 Moving Away from Object Detection and Towards Local Features Authors approach does not rely on first detecting objects Instead low level features that are invariant to degradations such as scale and orientation are used Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 10 45 Outline 1 Background Current Image Search Methods Applying Object Detection to Image Search Detecting and Applying Low Level Features 2 Approach and Algorithm Reranking Results From Image Search Computing Low Level Features From Features to Similarity From Similarity to Centrality From Centrality to Reranking 3 Experimental Results User Evaluation Subjective Click Study Objective Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 11 45 Goal Find Common Features Among Images Difficulties Common features may be in any orientation rotated anywhere in the image at any scale relative size not the main focus of the image in the background a non standard color Figure Similarity measurement must handle potential rotation scale and perspective transformations Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 12 45 Outline 1 Background Current Image Search Methods Applying Object Detection to Image Search Detecting and Applying Low Level Features 2 Approach and Algorithm Reranking Results From Image Search Computing Low Level Features From Features to Similarity From Similarity to Centrality From Centrality to Reranking 3 Experimental Results User Evaluation Subjective Click Study Objective Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 13 45 Task from User Perspective What does the user want 1 Use existing Google image search algorithm to find the top k images given a verbal query 2 Rerank top k images to maximize relevance and diversity Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 14 45 Goal of Image Search Engines Retrieve image results that are relevant Retrieve image results that are diverse enough to cover variations of visual or semantic concepts Here Reduce top 1000 to representative 10 Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 15 45 Graph Model Model the imaginary user behavior given the visual similarities of the images to be ranked Treat images as web documents Treat similarities as probabilistic visual hyperlinks vislinks Estimate the probability of images being visited by a user following these vislinks Images with more estimated visits are ranked higher Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 16 45 Vislinks Versus Real Links Related web documents are connected by manually defined hyperlinks For images authors compute vislinks explicitly as a function of visual similarities Idea a user views one image other similar images may also be of interest If image u has a vislink to image v then there is some probability that the user will jump from u to v I Random Surfer Model Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 17 45 Outline 1 Background Current Image Search Methods Applying Object Detection to Image Search Detecting and Applying Low Level Features 2 Approach and Algorithm Reranking Results From Image Search Computing Low Level Features From Features to Similarity From Similarity to Centrality From Centrality to Reranking 3 Experimental Results User Evaluation Subjective Click Study Objective Dafna Bitton May 20 2008 CSE 291 PageRank for Product Image Search 18 45 Similarity Measurement Global features such as color histograms
View Full Document
Unlocking...