Learning Object Categories from Google s Image Search Introduction Contributions TSITSI pLSA a translation and scale invariant pLSA model Unsupervised learning on training set collected from Google image search and therefore unlabeled Related work Fergus Perona Perona FeiFei Fei Fei Zisserman Presented by Sudheendra Discovering objects and their locations in images Sivic Sivic Russell Efros Efros Zisserman Zisserman Freeman pLSA for object category recognition and segmentation A visual category filter for Google images Fergus Perona Perona Zisserman Reranking of Google images by learning a model Introduction Overview Main challenge Noisy training data Less than 15 of the images returned by Google are related to the keyword Large variations in scale position and pose Idea Build pLSA model with a number of topics Visual words of an image will fall under a common topic Visual words of positive examples will be similar Find this topic using a validation set of less noisy data pLSA pLSA Learning Using EM E step estimate P z d w M step update P z d and P w z Associates z with the image and feature Visual words from an image tend to fall under the same topic Eg Face Recognition Fix P w z and estimate P z d using EM Generative model Choose document image d with probability P d P d Choose topic z with probability P z d P z d Choose word with probability P w z P w z Thus Drawbacks Spatial information is not used Multiple instances of a category cannot be captured 1 ABS pLSA ABS pLSA Quantize the image into X bins Include spatial location with word to produce topic variable EM steps similar to pLSA Eg Airplane Generative model summary Choose document d with probability P d P d Choose topic z with probability P z d P z d Choose word w location x with probability P w x z P w x z Thus Drawback TSI pLSA Uses absolute location of feature Not translation or scale invariant TSI pLSA Location of feature calculated with respect to object centroid Airplane Generative model summary Choose document d with probability d Choose topic z with probability P z d P z d Choose word w relative position x over all possible values of centroid c with probability P w x z P w x z calculated as x scale and yy scale along with the centroid specify the bounding box of the object A grid of Xfg locations within the bounding box and one background bin for feature location Object centroid and scale captured in 44 vector latent variable c Marginalizing over the entire range of c is not feasible TSI pLSA Small set of c values estimated during learning and recognition Some Issues Learning Estimating c values Standard pLSA run on the training set k 1 K gaussians fitted to the locations of features weighted by p w z to obtain k values of c centroid centroid mean scale variance Captures multiple instances of objects in image Selecting the final classifier EM as before with P w P w x z estimated by marginalizing over above found c values Estimating c values Similar to above Visual words of positive examples should belong to a common topic topic Validation set will perform best under this common topic Selecting the number of topics Z Recognition used to to weight gaussians EM Lock P w z and iterate to find P z d summed over the found c values Chosen empirically Roc vs number of topics plotted for best topic under validation set and and best topic under test set 2 Datasets 700 regions per image using 4 different region detectors Training Google dataset Images automatically downloaded from Google image search using the category name Validation set first five images from image search in 7 different languages Other Manually gathered frames from Caltech and Pascal datasets Testing Parameters Manually gathered frames from Caltech and Pascal datasets Because the method requires large number of data for parameter estimation SIFT descriptor of 72 dimensions Larger histogram bins more appropriate for object categorization K means clustering with k 350 to obtain 350 visual words Number of grid positions Xfg 37 Number of topics Z 8 Experiments and Results Experiment Experiment 1 2 standard Google datasets data Training Caltechimages from Google image50search 50 training images from 50 8 topics and best topic chosen Caltech 2 topics using Google validation set No clear winner TSI pLSA performs better than Pascal the other methods in all categories exceptForeground Guitar and background ABS pLSA TSI pLSA imagesand combined intoare onenot rotation invariant training set unsupervised Experiments and Results Experiment 3 learning 6 topics and best chosen using performance on foreground only images TSITSI pLSA performs better than the other methods Comparison with other supervised methods TSITSI pLSA is slightly worse than the other methods but it is unsupervised Experiment 4 Improving Google s Image search Best topic from 8 topics trained on raw Google data Conclusions All three methods work on unlabeled Google dataset and automatically collected validation set and TSITSI pLSA performs best TSITSI pLSA identifies multiple instances of objects in images Can be used to rank images returned by Google TSITSI pLSA performs badly when objects are rotated 3
View Full Document
Unlocking...