Hand recognition and trackingAbstractHand recognition and trackingDongbin LeeECE in Clemson University, [email protected] method of recognizing objects for hand in images is proposed. First, the initialization of thevisual tracking system is critical in the performance of the real time system, but not much isknown about how it is done. In this project, the initialization problem of the visual trackingsystem is addressed, and a simple algorithm based on the image difference is suggested. Nextdetecting the motion of images, I wanted to track image contour, “shape” of the hand image.1. IntroductionI would like to solve two problems in this project is to identify the objects in a static sceneand to follow an object as it moves in image sequence to understand what a moving object isdoing. So, try to solve these problems in certain special cases. In the particular special casesthat I work on, I restrict the problem to that of tracking or identifying a known class of objects.I am interested in finding solutions that work for backgrounds about which we know nothing,and which might themselves be moving. The visual recognition and tracking over imagesequences has been studied a lot because it has numerous application in the real-time computervision system, and there are not a few successful algorithms available and commerciallyimplemented. Also some of them perform quite well even in the severe visual clutter [1, 2].However, A few of the algorithms [3] discuss explicitly how the initialization is done althoughit is critical in the performance of the real-time system. The initialization problem is describedas follows. Given incoming image sequence or video stream, we want to find an efficientalgorithm that produces a rough estimate of the configuration of the target, which can be usedas an initial estimate of the state for the visual tracker. The initialization problem can bedecomposed into two sub-problems; Object Localization and Motion Detection. The systemshould be able to detect if there is any apparent motion in the current frame.After detecting the motion, Tracking system is also important in image sequence or videostream. While Kalman filtering based trcking system only carries the first two moment of thedensity, so it is inadequate because it is based on Gaussian densities being unimodal whichcannot represent simultaneous alternative hypothesis. Another method, “Snakes” wereintroduced by Kass et.al to perform robust segmentation and region tracking by modeling anobject using outline contour information on the curvature of the contour and the motion ofobject, but it is very sensitive to the approximation of the coefficients. In this paper, a simpletracking system is implemented based on the CONDENSATION(CONditional DENSitypropagATION)-based algorithm in [1] as a Active Shape Model(ASM), which has recentlybeen derived independently by several researchers[4] with the autoregressive process motionmodel frequently used in Kalman filter and by detecting motion using differences betweenimage sequences, and displayed the object as a curvefit. So it shows very simple, robust, andan efficient algorithm in tracking system in mixed cluttered images or video streams.2. Object RecognitionIn order to understand what the pattern theory of image sequence is all about, it is necessaryto begin to make inferences about where the objects are. Figure 1 can be analysed to find outwhere a mouse or hand or baseball is likely to be.2.1 Shape Space Representation for a targetThere exists a more compact representation of an object contour, which has lowerdimensions and allows for only meaningful deformation. This representation of an objectFigure 1. Mixtured image with cluttercontour is called “shape-space” representation [1,2,3] using Condensation-based algorithm. Ashpae-space (W,0QWXQ ) is a linear mapping of a shape-space vector XNRX  to spline vectorQNRQ  and can obtained from the following equation.0QWXQ where Q, W, X and 0QWXQ  are defined in the following and Q and Wcan be expressed asthe below equations.yxQQQ00,001000010000xyyxQQQQWQ is the spline vector that describes the basic shape of an object implemented by curve-fitting and the shape vector which denote X(W). 0QWXQ  is usually drawn by hand as template ofcontrol point vector . The shape for a given target will be noted by shape space W, which is alinear parameterization of the set of allowed defomations of a base curve, and which can beobtained from 0QWXQ  considering possible motion model of the contour such as PDM. In thisproject, the affine model is used and, thus, the dimensionality of the shape-space vector X isonly six. A couple of examples of various configuration represented by X, estX in the nextestimated vector, are shown in figure 3 as an original image in [1]. There are a couple ofadvantages of this representation. First, this representation enforces the smoothness inherent inthe object contour. Second, this turns out to be more robust to measurement noise than edge-based representation, and it reduces the dimensionality considerably.Fig.2(L) Image difference in a mouseFig.3(R) Shape-space vectorsestX in [1] However, the condensation algorithm has been shown to be asymptotically correct as N,but the control point representation(Q) is not simple enough to maintain over time since thenumber of control points (NQ) is quite big for the most of objects, especially in hand. Also, itallows for the arbitrary deformation of the contour, which does not happen for any real object. 2.2 Object Localization and Motion DetectionFigure 2 shows the difference of the two adjacent images for a mouse using SSD(Sum ofSquared Difference). Intuitively, the boundaries orthogonal to the moving direction formridge-like regions in the difference image. These regions tend to form very narrow ridge-likestructure. After the regions are extracted using simple threshold method, the centroid of theregions should be found. One heuristic used to find the centroid is to find the centroid of theconvex hull of the boundary points. From this centroid and the convex hull points, the widthand height of the target is computed roughly. Once the object is inside the window as anapproximate object, then the object initialization and