CSCE 641 Computer Graphics: Animation with Motion CaptureSlide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8RotoscopingAnother exampleSlide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Video-based MocapSlide 27PlanningMovement Flowchart for GamesSlide 30Slide 31Processing MarkersSlide 33Slide 34Slide 35Slide 36Motion Capture Data FilesHuman Skeletal FileArticulated CharacterSkeletal ModelComposite 2D TransformationSkeletal Information and Default Pose from ASFJoint Angle Poses from AMC FilesHuman Skeletal ModelSlide 45Human Skeletal File (.asf)Individual Bone InformationSlide 48Slide 49Slide 50Slide 51Slide 52Slide 53Slide 54Root RepresentationSlide 56Slide 57Hierarchy/Bone ConnectionsSlide 59Slide 60Slide 61Slide 62Slide 63What Can We Do With .asf File?From Local Coordinate to World CoordinateSlide 66Slide 67Slide 68From Child to Parent NodeBone TransformSlide 71Slide 72Motion Data File (.amc)Slide 74Slide 75Composite 3D TransformationSlide 77Slide 78Slide 79Slide 80Slide 81Online Motion Capture DatabaseCSCE 641 Computer Graphics: Animation with Motion CaptureJinxiang ChaiOutline of Mocap (Motion Capture)Mocap historyMocap technologiesMocap pipelineMocap data fromatMotion Capture“ …recording of motion for immediate or delayed analysis or playback…”-David J. Sturman“The creation of a 3d representation of a live performance”- Alberto Menache“…is a technique of digitally recording movements for entertainment, sports, and medical applications.”- WikipediaHistory of Motion CaptureEadweard Muybridge (1830-1904)• first person to photograph movement sequencesHistory of Motion CaptureEadweard Muybridge (1830-1904)• first person to photograph movement sequences• whether during a horse's trot, all four hooves were ever off the ground at the same time. • the flying horseSequence of a horse jumping (courtesy of E. Muybridge)History of Motion CaptureEadweard Muybridge (1830-1904)• first person to photograph movement sequences• the flying horse• animal locomotion (20k pictures about men, women, children, animals, and birds).Woman walking downstairs (courtesy of E. Muybridge)RotoscopeAllow animators to trace cartoon character over photographed frames of live performances• invented by Max Fleischer in 1915RotoscopeAllow animators to trace cartoon character over photographed frames of live performances• invented by Max Fleischer in 1915• 2D manual motion captureA horse animated by rotoscoping from Muybridge’s photosRotoscoping“rotoscoping can be thought of as a primitive form or precursoro to motion capture, where the motion is ‘captured’ painstakingly by hand.” - SturmanMocap OverviewAnother exampleAllow animators to trace cartoon character over photographed frames of live performances• invented by Max Fleischer in 1915• 2D manual capture• the first cartoon character to be rotoscoped -- “Koko the clown”• the human character animation -- snow white and her prince (Walt Disney, 1937)Rotoscope“3D Rotoscoping”: measuring 3D positions, orientations, velocities or accelerationsCurrent motion capture systems• Electromagnetic• Electromechanical• Fiber optic • OpticalCurrent Motion Capture TechnologiesEach sensor record 3D position and orientationEach sensor placed on joints of moving objectFull-body motion capture needs at least 15 sensorsPopular system: http://www.ascension-tech.com/Electromagnetic MocapSee video demo [1, 2]!Electromagnetic MocapPros• measure 3D position and orientation• no occlusion problems• can capture multiple subjects simultaneously Cons• magnetic perturbations (metal)• small capture volume• cannot capture deformation (facial expression)• hard to capture small bone movement (finger motion)• not as accurate as optical mocap systemElectromagnetic MocapEach sensor measures 3D orientationElectromechanical MocapEach sensor measures 3D orientationEach sensor placed on joints of moving objectFull-body motion capture needs at least 15 sensorsPopular systems:http://www.xsens.com/Electromechanical MocapSee video demo [1,2]!Electromechanical MocapPros• measure 3D orientation• no occlusion problems• can capture multiple subjects simultaneously • large capture volume• Outdoors capture (e.g. skiing) Cons• getting 3D position info is not easy• cannot capture deformation (facial expression)• hard to capture small bone movement (finger motion)• not as accurate as optical mocap systemElectromechanical MocapMeasures 3D position and orientation of entire tapeBinding the tape to the bodyPopular systems: http://www.measurand.com/Fiber Optic MocapSee video demo [1,2]!Fiber Optic MocapPros• measure 3D orientation and position• no occlusion problems• can capture multiple subjects simultaneously • go anywhere mocap system• can capture hand/finger motionCons• intrusive capture • cannot capture deformation (facial expression)• not as accurate as optical mocap systemFiber Optic MocapMultiple calibrated cameras (>=8) digitize different views of performanceWears retro-reflective markersAccurately measures 3D positions of markersOptical MocapSee video demo [1,2]!Optical MocapVicon mocap system: http://www.vicon.comPros• measure 3D position data also orientation• the most accurate capture method• very high frame rate• can capture very detailed motion (body, finger, facial deformation, etc.)Cons• has occlusion problems• hard to capture interactions among multiple ppl• limited capture volumeOptical MocapVideo-based Mocap•Mocap using a video camera, click here.•Motion capture using kinects, click hereMocap PipelineOptical Mocap pipeline• Planning• Calibration• Data processingPlanning•Motion capture requires serious planning•Anticipate how the data will be used•Garbage in garbage out•Shot list•Games–motions need to be able to blend into one an another–capture base motions and transitions–which motions transition into which other transitions–cycles/loopsMovement Flowchart for Games•Planning and Directing Motion Capture For GamesBy Melianthe Kines GamasutraJanuary 19, 2000URL: http://www.gamasutra.com/features/20000119/kines_01.htmPlanningCharacter/prop set up - character skeleton topology (bones/joints number, Dofs for each bone) - location and size of propsMarker Setup - the number of markers - where to place markersCalibrationCamera Calibration: determine the location and