Lecture 17: Recognition IIITuesday, Nov 13Prof. Kristen GraumanOutline• Last time:– Model-based recognition wrap-up– Classifiers: templates and appearance models• Histogram-based classifier• Eigenface approach, nearest neighbors• Today:– Limitations of Eigenfaces, PCA– Discriminative classifiers• Viola & Jones face detector (boosting)• SVMsImages (patches) as vectorsSlide by Trevor Darrell, MITOther image features– vector of pixel intensities– grayscale / color histogram– bank of filter responses Other image features– vector of pixel intensities– grayscale / color histogram– bank of filter responses – SIFT descriptorOther image features– vector of pixel intensities– grayscale / color histogram– bank of filter responses – SIFT descriptor–bag of words…Feature space / RepresentationFeature dimension 1Feature dimension 2Last time: Eigenfaces• Construct lower dimensional linear subspace that best explains variation of the training examplesPixel value 1Pixel value 2u1A face imageA (non-face) imageLast time: Eigenfaces• Premise: set of faces lie in a subspace of set of all images• Use PCA to determine the k (k<d) vectors u1,…ukthat span that subspace:x =~ μ + w1u1+ … + wkuk• Then use nearest neighbors in “face space” coordinates (w1,…wk) to do recognitiond = num rows * num cols in training imagesLast time: EigenfacesTraining images:x1,…,xNLast time: EigenfacesTop eigenvectors of the covariance matrix: u1,…ukMean: μPixel value 1Pixel value 2u1Last time: EigenfacesFace x in “face space” coordinates [w1,…,wk]: project the vector of pixel intensities onto each eigenvector.Last time: EigenfacesReconstruction from low-dimensional projection:+ =++++++Original face vectorReconstructed face vectorLast time: Eigenface recognition• Process labeled training images:– Unwrap the training face images into vectors to form a matrix– Perform principal components analysis (PCA): compute eigenvalues and eigenvectors of the covariance matrix– Project each training image onto subspace• Given novel image:– Project onto subspace– If Unknown, not face–ElseClassify as closest training face in k-dimensional subspaceBenefits• Form of automatic feature selection• Can sometimes remove lighting variations• Computational efficiency:– Reducing storage from d to k– Distances computed in k dimensionsLimitations• PCA useful to represent data, but directions of most variance not necessarily useful for classification Alternative: FisherfacesBelhumeur et al. PAMI 1997Rather than maximize scatter of projected classes as in PCA, maximize ratio of between-class scatter to within-class scatter by using Fisher’s Linear DiscriminantLimitations• PCA useful to represent data, but directions of most variance not necessarily useful for classification • Not appropriate for all data: PCA is fitting Gaussian where Σ is covariance matrixThere may be non-linear structure in high-dimensional data.Figure from Saul & RoweisLimitations• PCA useful to represent data, but directions of most variance not necessarily useful for classification • Not appropriate for all data: PCA is fitting Gaussian where Σ is covariance matrix• Assumptions about pre-processing may be unrealistic, or demands good detectorPrototype faces• Mean face as average of intensities: ok for well-aligned images…Mean: μ…but unaligned shapes are a problem.Prototype facesWe must include appearance AND shape to construct a prototype.Prototype faces in shape and appearanceUniversity of St. Andrews, Perception Laboratory Figures from http://perception.st-and.ac.uk/Prototyping/prototyping.htmMark coordinates of standard featuresCompute average shape for a group of facesWarp faces to mean shape. Blend images to provide image with average appearance of the group, normalized for shape.Compare to faces that are blended without changing shape.123Using prototype faces: agingBurt D.M. & Perrett D.I. (1995) Perception of age in adult Caucasian male faces: computer graphic manipulation of shape and colour information. Proc. R. Soc. 259, 137-143.Shape differences for 25-29 yr olds and 50-54 yr oldsAverage appearance and shape for different age groups.Using prototype faces: agingBurt D.M. & Perrett D.I. (1995) Perception of age in adult Caucasian male faces: computer graphic manipulation of shape and colour information. Proc. R. Soc. 259, 137-143.Enhance their differences to form caricatureCaricature“Facial aging”: get facial prototypes from different age groups, consider the difference to get function that maps one age group to another.University of St. Andrews, Perception LaboratoryUsing prototype faces: agingBurt D.M. & Perrett D.I. (1995) Perception of age in adult Caucasian male faces: computer graphic manipulation of shape and colour information. Proc. R. Soc. 259, 137-143.• http://morph.cs.st-andrews.ac.uk//Transformer/Aging demoBabyInputChildTeenager Older adult“feminize”BabyChildTeenager Older adult• http://morph.cs.st-andrews.ac.uk//Transformer/Aging demoInput“Masculinize”Outline• Last time:– Model-based recognition wrap-up– Classifiers: templates and appearance models• Histogram-based classifier• Eigenface approach, nearest neighbors• Today:– Limitations of Eigenfaces, PCA– Discriminative classifiers• Viola & Jones face detector (boosting)• SVMsLearning to distinguish faces and “non-faces”• How should the decision be made at every sub-window?Feature dimension 1Feature dimension 2Learning to distinguish faces and “non-faces”• How should the decision be made at every sub-window?• Compute boundary that divides the training examples well…FACE NON-FACEFeature dimension 1Feature dimension 2Questions• How to discriminate faces and non-faces?– Representation choice– Classifier choice• How to deal with the expense of such a windowed scan?– Efficient feature computation– Limit amount of computation required to make a decision per window[CVPR 2001]Value at (x,y) is sum of pixels above and to the left of (x,y)Can be computed in one pass over the original image:Defined as:Value at (x,y) is sum of pixels above and to the left of (x,y)Defined as:Large library of filters180,000+ possible features associated with each image subwindow…efficient, but still can’t compute complete set at detection time.Boosting• Weak learner: classifier with accuracy that need be only better than chance– Binary
View Full Document
Unlocking...