FACE DETECTION THROUGH TEMPLATE MATCHING AND COLOR SEGMENTATION By Scott Tan Yeh Ping ([email protected]) Chun Hui Weng ([email protected]) Boonping Lau ([email protected]) (EE 368 Final Project)FACE DETECTION THROUGH TEMPLATE MATCHING AND COLOR SEGMENTATION Scott Tan Yeh Ping, Chun Hui Weng, Boonping Lau ABSTRACT A process for face detection, which involves multi-resolution template matching, region clustering and color segmentation, works with high accuracy, and gives good statistical results with training images. Given the generality of the images and the templates used, the assumption would be that the implementation works well on other images, regardless of the scene lighting, size of faces or type of faces in the pictures. INTRODUCTION The increasing use of computer vision in security in place of humans led many to research the problem of face detection in images. The problem is not a trivial one as the classification of a human face proves to challenging. Despite the many variations of a human face, features can still be found, given a certain context, which will uniquely identify a face. In this project, we attempt to come up with a technique to give a high percentage of face detection based on correlation and post-processing. The first part of this paper will concentrate on the approach used in solving this problem, namely: correlation, false/repeated hits removal techniques, color segmentation and multi-resolution approach. The second section will go into the actual results of the implementation, where statistics taken after performing the algorithm on training images will be provided. A conclusion will then be given.APPROACH The algorithm used for face detection in this project is given below: Template matching is performed first to find the regions of high correlation with the face and eyes templates. Subsequently, using a mask derived from color segmentation and cleaned by texture filtering and various binary operations, the false and repeated hits are removed from the template matching result. The output of this process is then passed to a clustering procedure, where points are within a certain euclidean distance from one another will be clustered into one point. The whole process will then be repeated at a different scale/resolution. The outputs from each resolution are then recombined into a single mask. The individual processes will be explained in detail below. COLOR SEGMENTATION The aim of colorspace transformation is to increase the separability between skin and non-skin classes while decreasing the separability among skin tones. Hopefully it will bring robust performance under varying illumination conditions. However, there are many colorspaces to choose from and a large number of metrics to judge whether they are effective. Some potential colorspaces that we were interested in : • CIEXYZ • CIEXYZ • YCbCr • YUV • YIQ Eye/Face Template Matching False/Repeated Hit Removal Clustering Color Segmentation Multi-Resolution ProcessingA performance metric that others have used includes the computation of the separability of clusters of skin and non-skin pixels using scatter matrices. Another is to do a histogram comparison of the skin and non-skin pixels after colorspace transformation. The YCbCr colorspace was found to perform very well in 3 out of the 4 peformance metrics used1, so we decided to use it in our color segmentation algorithm. Here is the equation for transforming from RGB to YCbCr. −−−−+=BGRCrCbY214.18786.93112112203.74797.37966.24553.128481.6512812816 To transform back… −−−=12812816000791071.000456621.000318811.000153632.000456621.000625893.0000456621.0CrCbYBGR 1 http://marathon.csee.usf.edu/~tsap/papers/wacv02.pdf Fig 1. Skin color distribution histogram in Cb-Cr plane based on pixels under ref*.pngThis is a plot of the skin distribution against Cr and Cb in the YCbCr colorspace. If the sample size is large enough, it will be a probability distribution for the human skin. However, our data was restricted to the training pictures we had and we did our best to find the best skin filter. Attempts were made to retrieve online databases for reference to generalize our skin filter, however it was found that the values that work best were still those derived from the training data at the loss of generalization. Many studies have come to the conclusion that pixels belonging to skin region have similar Cb and Cr values. It is also conjectured that the perceived difference in skin color cannot be differentiated from the chrominance information of that region. The fairness or darkness of the skin is characterized by the difference in the brightness of the color, which is determined by Y. Therefore solely using Cb and Cr will generalize the filter for people of all skin colors. Several approaches were tried to define a boundary for skin and non-skin pixels. The first one we tried was an elliptical approach. Basically, it was to try to find the best fitting ellipses to enclose the non-zero region we have above. Another approach we tried was to use a histogram model. Basically we built a 256x256 mapping function from (Cr,Cb) to the decision function of skin or non-skin. By thresholding, we have a set S of tuples of Cr, Cb values that will be considered as skin. If the number of skin pixels falling into that particular bin with a certain (Cr, Cb) value exceeds a certain threshold, we will consider that a skin color. f(Cr,Cb) = 1 if (Cr,Cb) ∈ S 0 if (Cr,Cb) ∉ S This approach is better than the elliptical approach in a few aspects. Basically, we can have a better bounding shape than an ellipse. However, it could overtrain the model we are trying to build. It is also computationally simpler than an ellipse since it just involves looking up in the map to see if the current pixel should be classified as skin. The output from the histogram model has to undergo a morphological operation. For this purpose, MATLAB’s imfill was used. It is a function that will change zeros to ones when the zeros are fully surrounded by ones.As can be seen from above, most of the skin pixels have been correctly