CSCE 441 Computer Graphics: Animation with Motion CaptureSlide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8RotoscopingAnother exampleSlide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Video-based MocapSlide 27Slide 28Slide 29PlanningMovement Flowchart for GamesSlide 32Slide 33Processing MarkersSlide 35Slide 36Slide 37Motion Capture Data FilesHuman Skeletal FileHuman Skeletal ModelSlide 41Human Skeletal File (.asf)Individual Bone InformationSlide 44Slide 45Slide 46Slide 47Slide 48Slide 49Slide 50Root RepresentationSlide 52Slide 53Hierarchy/Bone ConnectionsSlide 55Slide 56Slide 57Slide 58Slide 59What Can We Do With .asf File?From Local Coordinate to World CoordinateSlide 62Slide 63Slide 64From Child to Parent NodeBone TransformSlide 67Slide 68Motion Data File (.amc)Slide 70Composite 3D TransformationSlide 72Slide 73Slide 74Slide 75Slide 76Online Motion Capture DatabaseComputational PhotographyCSCE 441 Computer Graphics: Animation with Motion CaptureJinxiang ChaiOutline of Mocap (Motion Capture)Mocap historyMocap technologiesMocap pipelineMocap data formatMotion Capture“ …recording of motion for immediate or delayed analysis or playback…”-David J. Sturman“The creation of a 3d representation of a live performance”- Alberto Menache“…is a technique of digitally recording movements for entertainment, sports, and medical applications.”- WikipediaHistory of Motion CaptureEadweard Muybridge (1830-1904)• first person to photograph movement sequencesHistory of Motion CaptureEadweard Muybridge (1830-1904)• first person to photograph movement sequences• whether during a horse's trot, all four hooves were ever off the ground at the same time. • the flying horseSequence of a horse jumping (courtesy of E. Muybridge)History of Motion CaptureEadweard Muybridge (1830-1904)• first person to photograph movement sequences• the flying horse• animal locomotion (20k pictures about men, women, children, animals, and birds).Woman walking downstairs (courtesy of E. Muybridge)RotoscopeAllow animators to trace cartoon character over photographed frames of live performances• invented by Max Fleischer in 1915RotoscopeAllow animators to trace cartoon character over photographed frames of live performances• invented by Max Fleischer in 1915• 2D manual motion captureA horse animated by rotoscoping from Muybridge’s photosRotoscoping“rotoscoping can be thought of as a primitive form or precursor to motion capture, where the motion is ‘captured’ painstakingly by hand.” - SturmanMocap OverviewAnother exampleAllow animators to trace cartoon character over photographed frames of live performances• invented by Max Fleischer in 1915• 2D manual capture• the first cartoon character to be rotoscoped -- “Koko the clown”• the human character animation -- snow white and her prince (Walt Disney, 1937)Rotoscope“3D Rotoscoping”: measuring 3D positions, orientations, velocities or accelerations automaticallyCurrent motion capture systems• Electromagnetic• Electromechanical• Fiber optic • OpticalCurrent Motion Capture TechnologiesEach sensor record 3D position and orientationEach sensor placed on joints of moving objectFull-body motion capture needs at least 15 sensorsPopular system: http://www.ascension-tech.com/Electromagnetic MocapSee video demo [1, 2]!Electromagnetic MocapPros• measure 3D positions and orientations• no occlusion problems• can capture multiple subjects simultaneously Cons• magnetic perturbations (metal)• small capture volume• cannot capture deformation (facial expression)• hard to capture small bone movement (finger movement)• not as accurate as optical mocap systemsElectromagnetic MocapEach sensor measures 3D orientations - including 3D accelerometers, 3D gyros, and 3D magnetometers Electromechanical MocapEach sensor measures 3D orientationsEach sensor placed on joints of moving objectFull-body motion capture needs at least 15 sensorsPopular systems:http://www.xsens.com/Electromechanical MocapSee video demo [1,2]!Electromechanical MocapPros• measure 3D orientations• no occlusion problems• can capture multiple subjects simultaneously • large capture volume• portable and outdoors capture (e.g. skiing)Cons• getting 3D position info is not easy • the root positions is often measured with ultrasonic position sensors• cannot capture deformation (facial expression)• hard to capture small bone movement (finger motion)• not as accurate as optical mocap systemElectromechanical MocapMeasures 3D position and orientation of entire tapeBinding the tape to the bodyPopular systems: http://www.measurand.com/Fiber Optic MocapSee video demo [1,2]!Fiber Optic MocapPros• measure 3D orientations and positions• no occlusion problems• can capture multiple subjects simultaneously • go anywhere mocap system• can capture hand/finger motionCons• intrusive capture • cannot capture deformation (facial expression)• not as accurate as optical mocap systemFiber Optic MocapMultiple calibrated cameras (>=8) digitize different views of performanceWears retro-reflective markersAccurately measures 3D positions of markersOptical MocapSee video demo [1,2,3,4]!Optical MocapVicon mocap system: http://www.vicon.comPros• measure 3D positions and orientations• the most accurate capture method• very high frame rate• can capture very detailed motion (body, finger, facial deformation, etc.)Cons• has occlusion problems• hard to capture interactions among multiple actors• limited capture volume• expensiveOptical MocapVideo-based Mocap•Capturing 3D performance from single-camera video streamsVideo-based Mocap•Capturing 3D performance from single-camera video streams•Click video hereVideo-based MocapPros: - capturing human motion anytime, anywhere - very cheap - zillions of films, sports footage, and internet videos.Cons: - not a mature technology - quality is not as good as other capturing technologies.Mocap PipelineOptical Mocap pipeline• Planning• Calibration• Data processingPlanning•Motion capture requires serious planning•Anticipate how the data will be used•Garbage in garbage out•Shot list•Games–motions need to be able to blend into one an another–capture base motions and transitions–which motions transition into which other transitions–cycles/loopsMovement Flowchart for Games•Planning and Directing Motion