Facial Feature Detection and Gaze TrackingSameer Bhide, Arjun Seshadri, and Brian WilliamsClemson University, ECE 847Abstract.In this paper, weexamine arobust, fast,a ndcheap schemefor locatingfacial featuresand tracking thegaze of a humaneye. Manyother eye-tracking andgaze-trackingmethods havebeen presentedearlier, but thesemethods are notboth robust andfast. Themethodproposed usescomplete graphmatching fromthresholdedimages forfacial featureidentification,and LongestLine Scanningand OccludedCircular EdgeMatchingtechniques foreye tracking.I.Introduction.Manyalgorithms existin digital imageprocessing toidentify pointsof interest (orfeature points)on an object andtrack them.However facialfeature detectionand gazetracking are notcommon point-trackingproblems. Theimplications ofthe solutions ofsuch problemsare far reaching,and couldprovide valuableassistance to thehandicapped,teachers, forgaming, and topsychologicaland roboticresearch.Gaze detectionallows acomputer to notonly knowwhere a personis in a givenenvironment,but to also knowwhat that personis presentlyinterested in orconcerned with.Successfulimplementationof facial featuredetection andgaze trackingwill yield veryfew ultimateconstraints, suchas when asubject’s eyesare closed orwhen the subjectturns away fromthe camera.There are alsoimplicationsreaching into AI.As might be thecase withresearch towardspsychology, acomputer couldbe programmedto consider whya subject islooking in acertain direction.Finally it is notunthinkable thataccurate gazedetection couldbe used as acontrol devicefor computers,quicklyreplacing themouse andkeyboard.II. FacialFeatureDetection.Beforeconsidering theimage of thehuman face, it isimportant to firstconsider thebackground ofthe image. Ifthe backgroundis cluttered ornoisy, we mayfind that ourdetectionalgorithm willrun into trouble.So let usconsider animage with anunobtrusive andunclutteredbackground,such as theimage below.Chen Now that wehave anappropriateimage fordetection, wecan beginattempting tofirst identify thesubject’s eyes.We perform athresholdingalgorithm toachieve theimage below.We now have abinary image ofblocks which arecandidates foreyes.The firstalgorithm usedto rule outobvious falsecandidates isconcerned withthe size of theblocks. Weknow to someextent that thelength and widthof the eyeblocks will bewithin somegiven range. Sowe considereach block, andrule out thosewhich falloutside of thisrange.(ObviouslyChen’s hair willbe ruled out,along with hissuit and chin.)We then look atthe ratio ofhorizontal andverticaldistancescontained in theblocks.Assuming thatthe subject’shead is notsignificantlytilted to oneside, the ratio ofvertical length tohorizontallength will beless than one.The twoaforementionedconditions mustbe fulfilled forblocks to still beconsideredcandidates.We then want toconsideremployinggraph matchingto find the twoblocks which aremost similar toeach other.These will beour finalcandidates. Theaforementionedequations comefrom (1).Normal_sizerefers to theratio of the areasof two blocks(area being thenumber ofpixels.Normal_averagerefers to thesimilaritybetween twoblocks in theoriginal image.(We use theblock in thebinary image toreference pixelsin the originalimage, then lookat their grayscale value.)Normal_Aspect_Ratio is theratio betweenthevertical/horizontal ratios (asmentionedbefore) of eachblock. Once wehave comparedall the remainingblocks, the twoblocks whosesum of thesethree variables isclosest to threewill be our finaleye candidates.We then haveidentified theeyes.The nose andmouth are foundbased ongeometry. Wedefine a regionbelow the eyesto search for thenostrils and lipcorners. If ourthresholdingalgorithm isgood enough,then we find thetwo nostrilseasily bysearching for thecandidate blockswhich are firstto spring upbelow the eyes. After the nostrilsare found, welocate thecorners of themouth asmaximum andminimumhorizontal pixelsfrom thecandidate block.III. GazeTracking. To study gazetracking, we setup anexperiment suchthat a subjectwas asked tolook at a grid of9 boxes (a 3x3grid,approximately 2feet in lengthand width).With the camerafocused solelyon the subject’seye, we shotcalibrationframes of thesubject lookingat each portionof the grid. To identify thesubject’s IRIS,we first employan edgedetectionalgorithm byway of a Cannyfilter. (A Sobelfilter was triedas well.) Wethenapproximate theiris by fitting asbest we can acircle to it.However toapproximate asubject’s gazewe must find thecenter of the iris.We utilize twoalgorithmspresented in (2)to do this. Thefirst is calledLongest LineScanning (LLS).IV LongestLine Scanning(LLS).LLS is aalgorithm basedon the followinguseful property:the center of anellipse lies onthe center of thelongesthorizontal lineinside theboundary of theellipse. Thoughwe do fit a circleto the iris, it willappear as anellipse in theimage if thesubject is notlooking directlyinto the camera.So we simplyscan the irisfrom top tobottom, eachtime plotting ahorizontal linebetween theedges. Thelongest linefound will giveus a candidatefor the center ofthe iris. SeeFigure 1. below.We take thecandidate centerfrom the LLSalgorithm, anduse it as an inputto our nextalgorithm,OccludedCircular EdgeMatching.V. OccludedCircular EdgeMatching(OCEM).Unfortunately,the LLSalgorithm maynot be ideal dueto the presenceof eyelids. Thelongest line mayactually behidden by thesubject’seyelids.Although theLLS methoddetects thecenter of the iris,the followingproblems arise:intra-iris noiseand rough irisedge.Obviously, if theedge of the irisis noisy, thehorizontal linedrawn in LLSwill not beeasily defined.OCEM takesboth thecandidate centerof the iris