Face ModelingThe Power of AveragingFigure-centric averagesMore by Jason Salavon“100 Special Moments” by Jason SalavonComputing MeansImages as VectorsVector Mean: Importance of AlignmentHow to align faces?Shape VectorAverage FaceObjects must span a subspaceExampleSubpopulation meansDeviations from the meanSlide 16Manipulating Facial Appearance through Shape and ColorSlide 18Changing genderEnhancing genderChanging ageBack to the SubspaceLinear Subspace: convex combinationsThe Morphable Face ModelThe Morphable face modelIssues:Principal Component AnalysisPCA via Singular Value DecompositionSlide 29EigenFacesFirst 3 Shape BasisUsing 3D Geometry: Blinz & Vetter, 1999Face Modeling15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005Portrait of Piotr Gibas© Joaquin Rosales GomezThe Power of AveragingFigure-centric averagesAntonio Torralba & Aude Oliva (2002)Averages: Hundreds of images containing a person are averaged to reveal regularities in the intensity patterns across all the images.More by Jason SalavonMore at: http://www.salavon.com/“100 Special Moments” by Jason SalavonWhy blurry?Computing MeansTwo Requirements:•Alignment of objects•Objects must span a subspaceUseful concepts:•Subpopulation means•Deviations from the meanImages as Vectors= mnn*mVector Mean: Importance of Alignment= mnn*m= n*m½ + ½ = mean imageHow to align faces?http://www2.imm.dtu.dk/~aam/datasets/datasets.htmlShape Vector=43Provides alignment!Average Face1. Warp to mean shape2. Average pixelshttp://graphics.cs.cmu.edu/courses/15-463/2004_fall/www/handins/brh/final/Objects must span a subspace(1,0)(0,1)(.5,.5)ExampleDoes not span a subspacemeanSubpopulation meansExamples:•Happy faces•Young faces•Asian faces•Etc.•Sunny days•Rainy days•Etc.•Etc.Average maleAverage femaleDeviations from the mean--==Image XMean XX = X - XDeviations from the mean+=+ 1.7=XX = X - XManipulating Facial Appearance through Shape and ColorDuncan A. Rowland and David I. PerrettSt Andrews UniversityIEEE CG&A, September 1995Face ModelingCompute average faces (color and shape)Compute deviations between male and female (vector and color differences)Changing genderDeform shape and/or color of an input face in the direction of “more female”original shapecolor bothEnhancing gendermore same original androgynous more oppositeChanging ageFace becomes “rounder” and “more textured” and “grayer”original shapecolor bothBack to the SubspaceLinear Subspace: convex combinationsmiiiXaX1Any new image X can beobtained as weighted sum of stored “basis” images. Our old friend, change of basis!What are the new coordinates of X?The Morphable Face Model The actual structure of a face is captured in the shape vector S = (x1, y1, x2, …, yn)T, containing the (x, y) coordinates of the n vertices of a face, and the appearance (texture) vector T = (R1, G1, B1, R2, …, Gn, Bn)T, containing the color values of the mean-warped face image. Shape SAppearance TThe Morphable face modelAgain, assuming that we have m such vector pairs in full correspondence, we can form new shapes Smodel and new appearances Tmodel as: If number of basis faces m is large enough to span the face subspace then:Any new face can be represented as a pair of vectors (1, 2m)T and (1, 2m)T !miiimodela1SSmiiimodelb1TTIssues:1. How many basis images is enough?2. Which ones should they be?3. What if some variations are more important than others?•E.g. corners of mouth carry much more information than haircutNeed a way to obtain basis images automatically, in order of importance! But what’s important?Principal Component AnalysisGiven a point set , in an M-dim space, PCA finds a basis such that•coefficients of the point set in that basis are uncorrelated•first r < M basis vectors provide an approximate basis that minimizes the mean-squared-error (MSE) in the approximation (over all bases with dimension r)x1x0x1x01st principal component2nd principal componentPCA via Singular Value Decomposition[u,s,v] = svd(A);http://graphics.cs.cmu.edu/courses/15-463/2004_fall/www/handins/brh/final/Principal Component AnalysisChoosing subspace dimension r:•look at decay of the eigenvalues as a function of r•Larger r means lower expected error in the subspace data approximationr M1eigenvaluesEigenFacesFirst popular use of PCA on images was for modeling and recognition of faces [Kirby and Sirovich, 1990, Turk and Pentland, 1991]Collect a face ensembleNormalize for contrast, scale, & orientation. Remove backgroundsApply PCA & choose the first N eigen-images that account for most of the variance of the data.mean facelighting variationFirst 3 Shape BasisMean appearancehttp://graphics.cs.cmu.edu/courses/15-463/2004_fall/www/handins/brh/final/Using 3D Geometry: Blinz & Vetter, 1999 show SIGGRAPH
View Full Document