Chrysalis A Single Source Video Based Motion Capture System John Li ([email protected]) Jung Hyun Yoon ([email protected]) Camillo J Taylor 4/10/2007Forward Forward Forward Forward ---- a personal message from the developers a personal message from the developers a personal message from the developers a personal message from the developers This paper and project represent the culmination of 1 year’s worth work by John and Jung. This project is also the last academic project for John and Jung after having worked together throughout college in such classes as CSE 371 and CSE 380. While we are very proud of all the work that has gone into this project and the outcome of the project, we are prouder to have had the opportunity to work together. Through the projects and the years, we both have enjoyed great success but have also been guilty of yelling at each other, some times well deserved and others not so. John served as Jung’s relationship counselor multiple times and Jung has laughed at John’s futile attempts at acting cool equally many times. Therefore, at this juncture before diving into the paper, the developers would like to thank each other for being a great partner but a better friend. Thank youThank youThank youThank you1. Abstract In the last few years, many researchers have studied human motion for the purpose of digital 3D reconstruction. A wide spectrum of commercial applications such as character animation, games, sport performance analysis, and surveillance systems motivate this research. However, research focuses typically on either special cameras [1] or special outfits with embedded sensors [2]. These solutions usually demonstrate a high degree of accuracy and reliability at significant cost. Ideally, the capture of motion data should be easily available and affordable for the typical computer user. In that sense, developing a tracking system that can render from a single video source is an enticing idea. The Chrysalis project seeks to develop just such a simple, inexpensive, and accurate video motion capture system. This paper will discuss how the Chrysalis project can develop from a simple avi video file, the type created from any retail digital camera, a 3D reconstruction of the human figurers in the video. In developing such a system, the developers needed to overcome several significant challenges such as the non-linear movement of limbs, ambiguities in mapping from 2D image measurements to 3D model configurations and self occlusion problems. The Chrysalis project makes several important assumptions in overcoming these difficulties. First, the relative bone lengths must be known a priori. Second, the scope of rendering is limited to a square mat whose vertices are pinpointed by the user. Lastly, the subject must be of significant distance away from the camera so that the self-occlusion problem causes minimal effects. The paper will discuss in greater detail later the importance of each assumption and their specific impacts. In comparison to previous work on this subject, the Chrysalis project makes one important improvement. As discussed earlier, the Chrysalis system seeks to make motion capture as simple and as inexpensive as possible. The project does this by simplifying the interface as well as the requirements needed to use the system. For example, the project does not use any additional external equipment besides a simple retail digital camera. Although the project does not develop rendering in as much detail as available retail solutions, the project at its core creates a well developed base for accurate motion capture and future development. 2. Related Work Current Computer Vision literature proposes a variety of techniques for motion capture. For example, Bregler [8] presents an impressive motion capture method involving updating kinematic chain parameters by using the product of exponential maps and twist motions. While Gavrila [5] presents a vision system for the 3D model based on the tracking of unconstrained human movement using image sequences acquired simultaneously from multiple views. Dockstader [6], on the other hand, proposes a real-time computing platform for tracking multiple interacting persons in motion using a multi-view implementation, whichcombats the negative effects of occlusion and enhances accuracy and reliability. Theobalt [3] and Bottino [4] both describe non-intrusive motion capture method schemes, which extract the actors’ silhouettes from synchronized video frames via background segmentation and determine sequences of 3D poses for human models. Davis [7] proposes an interface for rapidly creating 3D figure animation from 2D sketches of the character by coupling a user-guided pose reconstruction algorithm with optimization. This interface allows the user to choose from possible 3D configurations and refine the pose. Finally Sidenbladh [9] proposes a probabilistic method for tracking articulated human figures using monocular video. Within a Bayesian frame work, he defines a generative model in terms of the shape, appearance, and motion of the body, which leads to a likelihood function that specifies the probability of observing an image given the model parameters. However, these motion capture systems all suffer from some serious limitations. For example, considering most individuals only record motion sequences with a single camera, Dockstader’s method suffers from the flaw of complexity. Meanwhile, while a silhouette-based motion estimation method allows for certain levels of simplicity it still suffers from the same limitation. A Silhouette-based approach is particularly useful for long video sequences since the users do not have to input configuration information in each frame. However, this approach is also based on multiple camera viewpoints. In conclusion, while significant research has been done on the subject of motion capture, more than this paper can possibly adequately give credit to, most research tends to focus on complex solutions that are beyond the scope of a casual user. Finally in developing this project, the developers must give credit to Anthony Crowley and his work on his VideoMoCap system a few years ago. Anthony’s VideoMoCap system shared