Detecting Faces in Images: A SurveyMing-Hsuan Yang, Member, IEEE, David J. Kriegman, Senior Member, IEEE,andNarendra Ahuja, Fellow, IEEEAbstract—Images containing faces are essential to intelligent vision-based human computer interaction, and research efforts in faceprocessing include face recognition, face tracking, pose estimation, and expression recognition. However, many reported methodsassume that the faces in an image or an image sequence have been identified and localized. To build fully automated systems thatanalyze the information contained in face images, robust and efficient face detection algorithms are required. Given a single image, thegoal of face detection is to identify all image regions which contain a face regardless of its three-dimensional position, orientation, andlighting conditions. Such a problem is challenging because faces are nonrigid and have a high degree of variability in size, shape, color,and texture. Numerous techniques have been developed to detect faces in a single image, and the purpose of this paper is tocategorize and evaluate these algorithms. We also discuss relevant issues such as data collection, evaluation metrics, andbenchmarking. After analyzing these algorithms and identifying their limitations, we conclude with several promising directions forfuture research.Index Terms—Face detection, face recognition, object recognition, view-based recognition, statistical pattern recognition, machinelearning.æ1INTRODUCTIONWITH the ubiquity of new information technology andmedia, more effective and friendly methods forhuman computer interaction (HCI) are being developedwhich do not rely on traditional devices such as keyboards,mice, and displays. Furthermore, the ever decreasing price/performance ratio of computing coupled with recentdecreases in video image acquisition cost imply thatcomputer vision systems can be deployed in desktop andembedded systems [111], [112], [113]. The rapidly expand-ing research in face processing is based on the premise thatinformation about a user’s identity, state, and intent can beextracted from images, and that computers can then reactaccordingly, e.g., by observing a person’s facial expression.In the last five years, face and facial expression recognitionhave attracted much attention though they have beenstudied for more than 20 years by psychophysicists,neuroscientists, and engineers. Many research demonstra-tions and commercial applications have been developedfrom these efforts. A first step of any face processing systemis detecting the locations in images where faces are present.However, face detection from a single image is a challen-ging task because of variability in scale, location, orientation(up-right, rotated), and pose (frontal, profile). Facialexpression, occlusion, and lighting conditions also changethe overall appearance of faces.We now give a definition of face detection: Given anarbitrary image, the goal of face detection is to determinewhether or not there are any faces in the image and, ifpresent, return the image location and extent of each face.The challenges associated with face detection can beattributed to the following factors:. Pose. The images of a face vary due to the relativecamera-face pose (frontal, 45 degree, profile, upsidedown), and some facial features such as an eye or thenose may become partially or wholly occluded.. Presence or absence of structural components.Facial features such as beards, mustaches, andglasses may or may not be present and there is agreat deal of variability among these componentsincluding shape, color, and size.. Facial expression. The appearance of faces aredirectly affected by a person’s facial expression.. Occlusion. Faces may be partially occluded by otherobjects. In an image with a group of people, somefaces may partially occlude other faces.. Image orientation. Face images directly vary fordifferent rotations about the camera’s optical axis.. Imaging conditions. When the image is formed,factors such as lighting (spectra, source distributionand intensity) and camera characteristics (sensorresponse, lenses) affect the appearance of a face.There are many closely related problems of facedetection. Face localization aims to determine the imageposition of a single face; this is a simplified detectionproblem with the assumption that an input image containsonly one face [85], [103]. The goal of facial feature detection isto detect the presence and location of features, such as eyes,nose, nostrils, eyebrow, mouth, lips, ears, etc., with theassumption that there is only one face in an image [28], [54].Face recognition or face identification compares an input image(probe) against a database (gallery) and reports a match, if34 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 24, NO. 1, JANUARY 2002. M.-H. Yang is with Honda Fundamental Research Labs, 800 CaliforniaStreet, Mountain View, CA 94041. E-mail: [email protected].. D.J. Kriegman is with the Department of Computer Science and BeckmanInstitute, University of Illinois at Urbana-Champaign, Urbana, IL 61801.E-mail: [email protected].. N. Ahjua is with the Department of Electrical and Computer Engineeringand Beckman Institute, University of Illinois at Urbana-Champaign,Urbana, IL 61801. E-mail: [email protected] received 5 May 2000; revised 15 Jan. 2001; accepted 7 Mar. 2001.Recommended for acceptance by K. Bowyer.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log Number 112058.0162-8828/02/$17.00 ß 2002 IEEEany [163], [133], [18]. The purpose of face authentication is toverify the claim of the identity of an individual in an inputimage [158], [82], while face tracking methods continuouslyestimate the location and possibly the orientation of a facein an image sequence in real time [30], [39], [33]. Facialexpression recognition concerns identifying the affectivestates (happy, sad, disgusted, etc.) of humans [40], [35].Evidently, face detection is the first step in any automatedsystem which solves the above problems. It is worthmentioning that many papers use the term “face detection,”but the methods and the experimental results only showthat a single face is localized in an input image. In thispaper, we differentiate face detection from face localizationsince the latter is a simplified problem of the former.Meanwhile, we focus on face detection methods rather thantracking methods.While
View Full Document
Unlocking...