Copyright © 2005 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail [email protected]. © 2005 ACM 0730-0301/05/0700-0667 $5.00 Action Synopsis: Pose Selection and IllustrationJackie Assa Yaron CaspiSchool of Computer Science Tel Aviv UniversityDaniel Cohen-Or(a) (b) (c)Figure 1: Action synopsis analyzes the motion-curve embedded in a low dimensional space (a); super-positioning of carefully selected poses(b) vs. uniform sampling (c). (Images used courtesy of Moshe Mahler and Jessica Hodgins, copyright Carnegie Mellon University.)AbstractIllustrating motion in still imagery for the purpose of summary, ab-straction and motion description is important for a diverse spectrumof fields, ranging from arts to sciences. In this paper, we introducea method that produces an action synopsis for presenting motionin still images. The method carefully selects key poses based onan analysis of a skeletal animation sequence, to facilitate express-ing complex motions in a single image or a small number of con-cise views. Our approach is to embed the high-dimensional motioncurve in a low-dimensional Euclidean space, where the main char-acteristics of the skeletal action are kept. The lower complexity ofthe embedded motion curve allows a simple iterative method whichanalyzes the curve and locates significant points, associated withthe key poses of the original motion. We present methods for il-lustrating the selected poses in an image as a means to convey theaction. We applied our methods to a variety of motions of humanactions given either as 3D animation sequences or as video clips,and generated images that depict their synopsis.CR Categories: I.3.7 [Computer Graphics]: Three-dimensionalgraphics and realism—Animation.; I.3.3 [Computer Graphics]:Picture/Image generation—Displaying and viewing algorithms.Keywords: human motion analysis, animation analysis, key poses,dimensionality reduction, motion curve1 IntroductionExpressing motion in still images, sculptures and monuments hasintrigued artists and scientists alike for centuries. From AncientGreece to the 19th century, human motion was captured mainlythrough instants of motion, which were presented by such tech-niques as interesting asymmetric poses, or by use of artistic ele-ments such as hair and clothes swaying in the wind, which con-tributed to the sensation of movement [Ward 1979; Braun 1992].Pioneers in the field of Photography also addressed the challengeof expressing human motion as an image. A technique known asChronophotography captures a sequence of still images with a shorttime difference to allow displaying fast human activities. Stro-boscopy [Cutting 2002] is currently used as a photography tech-nique where the same film is used repeatedly at different time in-stances. The resulting image consists of a composition of frameswith predefined time intervals which might not successfully conveythe full motion, or perhaps cause self-occlusion.The need to display human activity in still images exists in manyfields of modern life. Illustrating a short sport event in a newspaper,or a dynamic experiment in a scientific journal, are examples ofthe need to display dynamic action in the printed media. Anotherexample is a thumbnail of a video clip or animation stored in digitallibraries. Typically, the first frame is used to represent the content.The method presented in this paper is a step toward improving theinformative values of still images to convey short actions.The method introduced in this paper, automatically selects posesthat best represent an action for creating an action synopsis, allow-ing expression of complex motions in a still imagery. The mainchallenge in our work is to carefully select a small number of keyposes taken from an animation sequence (3D) or a video clip (2D),and to present the associated frames in a concise view, in whichthe full motion is self explanatory. This is demonstrated in Figure1, where the four key poses, selected by our technique, convey theaction better than the frames sampled with uniform time intervals.Studies by Kovar et al. [2002; 2004] Park and Shin [2004], andLee et al. [2002] attempted selecting key-poses mainly for motionsynthesis and retrieval, however the number of key-poses they ex-667Figure 2: A locomotion series by Muybridge from the 19th centurycapturing human motion. Image courtesy of the Muybridge col-lection. Copyright of the National Museum of American History,Smithsonian Institution. http://www.si.edu/.tract is more than an order of magnitude higher than can be usedin an image composition such as ours. Similar work has attemptedto select key-frames from video sequences based on image spacecriteria [DeMenthon et al. 1998; Vermaak et al. 2002; Fauvet et al.2004]. Their approaches focus on the background scene and on thecamera motion. In our work we handle a different media - anima-tion sequences, focusing on a skeleton-based model activity.Action synopsis selects key poses based on an analysis of theskeletal motion curve. The raw data that represents the skeletonmotion is a high dimensional curve of poses, where each pose con-sists of the skeleton joints and their associated attributes or aspects.The key-idea is to successfully embed the high dimensional motioncurve in a low dimensional Euclidean space that represents the ac-tual motion well. The lower complexity of the embedded motioncurve allows a rather simple geometric tool to analyze the curve inorder to disclose significant points. The selected points along thecurve are associated with the key poses of the original motion. Thekey-poses or the associated frames are used to synthesize an imageor a small number of images for illustrating the action. The selec-tion of the key-poses allows the generation of a clear understandingof the action, in particular, reducing self occlusions, adding meansto express the action tempo, or selecting an