Efficient Generation of Motion Transitions using Spacetime Constraints

**Efficient** **Generation** **of** **Motion** **Transitions** **using** **Spacetime** **Constraints**Charles RosePrinceton University(currently at Micros**of**t Research)Brian GuenterMicros**of**t ResearchBobby BodenheimerMicros**of**t ResearchMichael F. CohenMicros**of**t ResearchAbstractThis paper describes the application **of** space time constraints to creatingtransitions between segments **of** human body motion. Themotion transition generation uses a combination **of** spacetime constraintsand inverse kinematic constraints to generate seamless anddynamically plausible transitions between motion segments. Weuse a fast recursive dynamics formulation which makes it possibleto use spacetime constraints on systems with many degrees **of** freedom,such as human figures. The system uses an interpreter **of** amotion expression language to allow the user to manipulate motiondata, break it into pieces, and reassemble it into new, more complex,motions. We have successfully used the system to create basis motions,cyclic data, and seamless motion transitions on a human bodymodel with 44 degrees **of** freedom.Additional Keywords and Phrases: computer animation, inversekinematics, motion capture, motion control, human figure animation,cyclification.CR Categories and SubjectDescriptions: I.3.7 [Computer Graphics]:Three Dimensional Graphics and Realism: Animation; I.6.3[Simulation and Modeling]: Applications; G.1.6 [Constrained Optimization];I.3.5 [Physically-Based Modeling].1 IntroductionExisting 3D animation tools primarily provide support for creating asingle linear stream **of** animation where the entire motion is plannedin advance and computed **of**f-line. Interactive 3D character animation,however, is characterized by a degree **of** uncertainty that isnot present in animation for film or television. Characters are underthe control **of** the user and must be able to change the way theymove at any time. Crafting animation for an interactive applicationpresents a new set **of** problems and requires a different set **of** specializedtools.One solution for these problems is to generate a set **of** high qualitymotions, called basis motions in the remainder **of** the paper, andthen create transitions between these motions so they can be strungtogether into animations **of** unlimited length and great variety. BasisEmail: ft-chuckr, briangu, bobbyb, mcoheng@micros**of**t.commotions are typically short and can be combined into motions representative**of** the type **of** actions that the body has to perform. Forexample, basis motions might be walk cycles, arm waves, karatekicks, and so forth. Basis motions need not specify all the degrees**of** freedom in the body; they can specify just those **of** one limb oreven a part **of** a limb. Using motion capture techniques, it is relativelyeasy to make high quality basis motions, but generating highqualitytransitions among those basis motions is still difficult andinvolves significant manual labor. The techniques presented workwell with motion capture data, but would work equally well withhand-animated basis motions.We have developed an algorithm for generating these transitionssemi-automatically, greatly reducing the time spent and the number**of** parameters an animator must specify. The system provides twosemi-automatic mechanisms for generating motion: motion transitiongeneration and cyclification. **Motion** transition generation usesa combination **of** spacetime constraints [10] and inverse kinematicconstraints [12] to generate transitions between basis motions. Afast dynamics formulation makes it practical to use spacetime transitiongeneration on high degree **of** freedom systems. With this dynamicsformulation, the algorithm achieves the lower bound timecomplexity for spacetime algorithms that use gradient based optimizationtechniques.The motion transitions satisfy both dynamic and kinematic constraints.This differs from the work described in [2, 11, 9]. Thesepapers described various mechanisms, such as dynamic time warping,Fourier interpolation, and multi-resolution signal processing,for transforming existing motion data. **Transitions** between motionclips were achieved **using** linear combinations **of** the two motionsbeing transitioned. This can result in motion which does not haverealistic dynamic qualities and which may not necessarily satisfykinematic or anthropomorphic constraints. The transition mechanismdescribed here generates motion which minimizes the torque requiredto transition from one motion to another while maintainingjoint angle constraints. Inverse kinematics are used to ensure thatkinematic constraints are satisfied.Additionally, we have defined a motion expression language toallow the user to manipulate motion data, break it into pieces, andreassemble it into new, more complex motions. We have successfullyused the system to create basis motions, cyclic data, and motiontransitions on our human body model which has 44 degrees **of**freedom.Section 2 **of** the paper describes the human body model and howmotion capture data is processed before it gets into the system.Section 3 describes the semi-automatic spacetime and inverse kinematictransition mechanism. This section also explains the fastdynamic formulation which allows spacetime constraint optimizationto run quickly on systems with many degrees **of** freedom. Section3 also describes our method **of** motion cyclification. Section 4explains how body motion is internally represented and describesthe motion expression language. Section 5 presents results **of** transitionsgenerated with the system. Section 6 concludes the paper.Appendix A contains an explanation **of** the notation used in the dynamicsformulation, the dynamics equations, and their partial derivativesas they are used in the spacetime optimization.

R is a function **of** the joint angles in the bodyR = f (q 1(t);:::;q n(t))which are themselves functions **of** the B-spline control points definingeach joint angle functionq i(t) =g(b i;1;:::;b i;m)where the b i;j are the control points **of** the B-spline curve for jointangle function q i(t). We minimize R **using** the BFGS optimizationalgorithm described in more detail in Section 3.2.3. For the purposes**of** the present discussion the only relevant part **of** the BFGSalgorithm is that it requires the gradient **of** R at each iteration in theoptimizationrR =@R@b i;1; ;@R@b i;j=@r k@b i;jZ t2t 1K Xk=1@R@b n;m@r k@b i;jdt=2(p k(t),^p k(t)) (u i d ki) @qi@b i;jwhere u i is the axis **of** rotation **of** joint i and d ki is the vector fromjoint i to the constrainedframe k. Figure 2 showsthe effect **using** theinverse kinematic constraint to fix the feet **of** the body on the groundduring a motion transition period. The left leg is constrained to beon the ground during the entire transition interval while the right legis constrained to be on the ground only at the end **of** the interval. Inthe image on the left the inverse kinematic constraints were turned**of**f; the left leg drifts from its desired position and the right leg failsto meet its desired position. In the image on the right the inversekinematic constraints are satisfied. The left leg remains fixed duringthe transition period and the right leg touches down at the end **of** theinterval.Figure 2: Effect **of** inverse kinematics constraint on placement **of**feet.3.2.3 **Spacetime** Dynamics FormulationThe energy required for a human to move along a path is actually acomplex non-linear function **of** the body motion since energy can bestored in muscle and tendon in one part **of** the motion and releasedlater on. As shown in [3], joint torques are a reasonable predictor **of**metabolic energy, so minimizing torque over time should be a reasonableapproximation to minimizing metabolic energy. Experiencehas shown that motion that minimizes energy looks natural. Thisleads to the minimization problem:minimize e =Z t2t 1Xi 2 i (t) dt:We use the BFGS optimization algorithm [5] to find a minimum**of** this integral equation. BFGS belongs to the class **of** quasi-Newton algorithms which progress toward a solution by **using** thegradient **of** the objective functiong = re:The gradient is used to incrementally update a matrix decomposition**of** a pseudo-Hessianmatrix, H, and to compute a new step directiond = ,H ,1 g:The relative amount **of** computation for each subtask required atevery iteration **of** the algorithm is common to several quasi-Newtonalgorithms: gradient computation, pseudo-Hessian update, andcomputation **of** the step direction.Since each **of** the i is potentially a function **of** all the q i, _q i, q i thegradient requires the evaluation **of** O(n 2 ) partial derivatives wheren is the number **of** degrees **of** freedom **of** the body. This is in facta lower bound for the asymptotic time complexity **of** space time algorithmswhich use gradient-based optimization techniques.If m is the number **of** B-spline coefficients used to define the timefunction **of** each degree **of** freedom then the pseudo-Hessian is **of**size nm by nm. The update **of** the pseudo-Hessianand computation**of** the step direction are both O((nm) 2 ).Form small, less than 20,and n large, more than 30, the time required to computeg dominatesall other computation thus an efficient formulation for g will pay thegreatest dividends in reducing computation.Computing g requires finding the joint torques and a variety**of** subsidiary quantities, such as angular velocity and acceleration.This is the inverse dynamics problem which has been extensivelystudied in the robotics literature. See [1] for a good overview **of**many **of** these algorithms. Many inverse dynamics formulationshave been proposed in the robotics literature ranging from O(n 4 )non-recursive to O(n) recursive algorithms. The inverse dynamicsformulation we use is due to Balafoutis [1]. This is an O(n)recursive formulation which requires 96n , 77 multiplications and84n,77 additions to solve the inverse dynamicsproblem for a robotmanipulator with n joints. This is faster than the O(n) Lagrangianrecursive formulation developed by Hollerbach [6] and used in [7],which requires 412n , 277 multiplications and 320n , 201 additions.The efficiency **of** the Balafoutis algorithm derives from the computationalefficiency **of** Cartesian tensors and from the recursivenature **of** the computations. These efficiencies carry over to thecomputation **of** the gradient terms.The Balafoutis algorithm proceeds in two steps. In the first stepvelocities, accelerations, net torques, and forces at each joint arecomputed starting from the root node and working out to the tips **of**all the chainsin the tree. In the secondstep the joint torques are computedstarting from the tips **of** the chains back to the root node. Seethe appendix for details. These recursive equations can be differentiateddirectly to compute g or one can use Cartesian tensor identitiesto compute the derivatives. Since the differentiation is tediousand somewhat involved we have included some **of** the partial derivatives**of** the recursive equations in the appendix as an aid to thoseattempting to reproduce our results.3.3 **Motion** CyclificationIf cyclic motions, such as walking and running, come from motioncapture data they will not be precisely cyclic due to measurementerrors and normal human variation in movement. The discontinuitiesin motion will likely be small enough that the full power **of** aspacetime transition will not be necessary in order to splice a motionback onto itself smoothly. In this case we use a much simpler andfaster algorithm for generating seamless cycles.

The cyclification algorithm proceeds in two steps. First the usermarks the approximate beginning and end **of** a cycle. We createtwo time regions I s and I f centered about the markers the user haschosen. The time regions are set to be one-fifth the length **of** the timebetween markers. The system then finds one time point in each intervalthat minimizes the difference between position, velocity, andacceleration **of** the body:mint 1 2I s;t 2 2I fka,bk 2where a =[q(t 1);_q(t 1);q(t 1)] T and b =[q(t 2);_q(t 2);q(t 2)] T .For most motions there will still be a discontinuity at the timewhere the motion cycles. We distribute this discontinuity error overthe entire time interval so that the end points **of** the cycle match exactlyby adding a linear **of**fset to the entire time interval. We thenconstructa C 2 motion curve by fitting a least squares cyclic B-splineapproximation to the modified motion.I 1^I 2 =I 1 , I 2 =((I 1 + I 2 = I 1 , I 2;I 1 ^I 2 t 1 t 2(f 1 ;:::;f n;g 1 ;:::;g n;t 3 ;t 2 ) t 3

above. Figure 6 shows an example **of** a spacetime transition from awalk arm motion to a wave arm motion and then back to a walk armmotion. The arm wave MU defines only the DOFs **of** the arm andthe walk MU defines all the DOFs **of** the body. First we perform anaffine transformation on the arm wave MU and undefine this fromthe walk MU. This will undefine the arm degrees **of** freedom **of** thewalk MU during a time that is slightly longer than the arm wave MU.When we add a time shifted version **of** the arm wave MU to the resultingMU there will be two undefined regions surrounding it whichthe spacetime operator will fill in with spacetime transition motion.The result will be a smooth transition from a walking arm motion toan arm wave motion and back to a walking arm motion. This operationis shown in both a time line view and an operator tree view inFigure 6. Letting SP denote the spacetime optimization, the algebraicrepresentation **of** this motion isWithout a biomechanical model to guide a large motion, our minimalenergy model will **of**ten prove insufficient.The beginning **of** the transition is colored blue and the end iscolored red with intermediate times a linear blend **of** the two colors.This motion is one transition from a longer animation which has 5transitions between 6 motions.SP (ane 2 (wave) + (walk , ane 1 (wave))) :**Spacetime**transitionregionsWalkWalkaffine2(wave)affine2(wave)WalkWalkSp+Figure 8: Multiple time exposure **of** transition generated from themotions in Figure 7Walkaffine2(wave)affine1 (wave)Walk-aff1aff2Figure 9 shows an example **of** a motion transition which affectsonly the arm degrees **of** freedom **of** the motion. This sequenceactuallyconsists **of** two space time transitions: one from a walking armmotion to the salute motion and another back to the walking armmotion. Each transition motion is .3 seconds long.WaveWaveWalkWalkTime line viewFunction composition viewFigure 6: **Spacetime** composition **of** motions.5 ResultsFigure 9: Arm walk motion transitioning to salute motion and backto walk motion. Arm degrees **of** freedom affected by the transitionare colored green.Figure 7: End position **of** motion 1 and beginning position **of** motion2 for a motion transitionWe have successfully applied the motion transition algorithm onmany motions. For this example the transition time was set to .6,and the number **of** B-spline coefficients to 5. The resulting transitionis shown in Figure 8. Our experience has been that successfultransitions are quite short, usually in the range **of** .3 to .6 seconds.Computation times for transitions are strongly dependent on thenumber **of** degrees **of** freedom involved since the spacetime formulationwe use is O(n 2 ) in the number **of** degrees **of** freedom. Forthe transition **of** Figure 8 generating the spacetime transition motiontook 72 seconds. This transition involved 44 degrees **of** freedom.For the transition **of** Figure 9 generating the spacetime transitiontook 20 seconds. All timings were performed on a 100 MHzPentium processor.**Spacetime** transitions are more costly to generate than joint angleinterpolation techniques, but they **of**ten produce more realistic motion.One type **of** motion that demonstrates this superiority is motionthat has identical joint space beginning and ending conditionson some **of** the degrees **of** freedom **of** the figure. An example **of** thistype **of** motion is shown in Figure 10. This motion begins with theforearm nearly vertical, held close to the shoulder with zero initial

AAppendix: Dynamics DetailsA.1 List **of** Symbolso ic iorigin **of** the i-th link coordinate frame.center **of** mass **of** the i-th link.! i i angular velocity **of** the i-th link.Figure 10: Joint angle interpolation versus spacetime optimization.z i is i i;jjoint axis **of** the i-thlink expressed in the i-thcoordinate frame.vector from o i to o j expressed in the i-th coordinate frame.velocity. The motion ends with the forearm held horizontal also withzero velocity. Because the upper arm and the wrist have identicaljoint space starting and ending conditions any simple interpolationtechnique, which would include linear interpolation, polynomial interpolation,and most other types **of** interpolation which simply takea weighted sum **of** the two endpoint conditions will yield a motionsuch as that shown on the left in Figure 10. This is an unnatural motionsince there is no joint space motion at the shoulder or the wrist.The spacetime motion, however, has motion at every joint and looksmuch more like the kind **of** motion a person might make.6 ConclusionThis paper has presented a powerful animation system for manipulatingmotion data **using** a motion expression interpreter. Data canbe positioned in space and time, and complete control **of** the degrees**of** freedom in the system allows motions to be spliced and mixed inarbitrary manners.The system is capable **of** generating seamless transitions betweensegments **of** animations **using** spacetime constraints and inverse kinematics.Using a new, fast, dynamics formulation we can applyspacetime constraints to systems having a large number **of** degrees**of** freedom and still have reasonable computation time. An additionalcapability **of** the system is the generation **of** arbitrary lengthperiodic motions, such as walking, by cyclifying segments **of** motiondata which are approximately periodic.The results **of** **using** our system to generate animations startingfrom a base library **of** soccer motions are quite good. Cyclification**of** a segment **of** such motions as a walk produces a quite realisticwalk **of** arbitrary length. The spacetime constraint and inverse kinematicoptimization produce transitions between diverse motionswhich are seamless and invisible to the viewer. While the optimizationcannot be done in real time, it is relatively fast, and quite usableby an animator designing motions.We plan to extend our motion model to more accurately modelthe dynamics **of** the human body model. The current approximationused for computing root motion works reasonably well for the class**of** transitions we have worked with but is not accurate for free bodymotion.AcknowledgementsThe authors thank Jessica Hodgins, College **of** Computing, GeorgiaInstitute **of** Technology,for providing inertia matrices for the humanbody model. The Micros**of**t Consumer Division generously sharedmotion capture data with us for use in this project.r i i;jA iI k c iJ k c i i iF i c iM i c ivector from o i to c j expressed in the i-th coordinate frame.3x3 coordinate (or 4x4 homogeneous) transformation relatingthe i-th coordinate frame to the (i , 1)-th frame.inertia tensor **of** the i-th link about c i expressed in the k-th coordinateframe.Euler’s inertia tensor **of** the i-th frame about c i expressed inthe k-th coordinate frame.angular acceleration tensor **of** the i-th link expressed in the i-thcoordinate frame.force vector acting on c i expressed in the i-th coordinateframe.moment vectoraboutc i expressedin the i-th coordinate frame.f i i force vector exerted on link i by link (i , 1). i i moment vector exerted on link i by link (i , 1). i torque at joint i.m ig gravity.mass **of** the i-th link.In the above, the subscript indicates the coordinate frame beingrepresented and superscript the coordinate frame in which it is represented.We use + and , on index variables to denote relative placementin the joint hierarchy. Thus, i, is the predecessor **of** i which is thepredecessor **of** i+. For example, in the equation ! i+z i+i+ = AT i+! i i +_qi+, the variable i+ !i i is the angular velocity in the coordinateframe which precedes the coordinate frame **of** ! i+ .Inotherwords,i+coordinate frame i is closer to the root coordinate frame than isframe i+. Note that there is no guarantee **of** a uniquely defined successor.g =[0:0;,9:80655; 0:0] TJ i c i = 1 trace, I i c2 i 1 , Iici" 0 ,v3 v 2dual(v) =~v= v 3 0 ,v 1,v 2 v 1 0dual(~v) =v#

A.2 Forward Dynamics EquationsBase conditions at the root **of** the creature:! 0 0 = z 0 0 _q 0_! 0 0 = z 0 0 q 0s 0 0;0 = A T 0 gRecursive forward dynamics equations:! i+i+ = A T i+! i i + z i+i+ _q i+_! i+i+ = AT i+ _! i i + ~! i+i z i+i+ i+i+ = ~! _ i+i+ + ~! i+i+ ~!i+ i+s i+0;i+ = A T i+ si0;i + i is i i;i+r i+0;i+ = i+ i+r i+i+;i+ + si+ 0;i+F i+c i+= m i+ r i+0;i+_qi+ + zi+i+ qi+~M i+c i+= , i+J i+c i+,, i+i+ J i+c i+ TA.3 Backward Recursive Equations (Torque Equations)At a joint controlling an end-effector:At an internal joint:f i i = F i c i+f i i = F i c i i i = ~r i i;iF i c i+ M i c i i = i i z i i i i = ~r i i;iF i c i+ M i c i+ i = i i z i iX Ai+f i+i+i+X Ai+ i+i+ + ~si i;i+f i i+A.4 The Energy Function and its PartialZe =tXi 2 i dti+R t 2@e @t= 1Pi 2 i dt=2@q j @q jZ t2t 1X i@ i@q jdtA.5 Forward Partials with Initial Conditions! i+i+ =i+>j A T ! i ii+q j q j! i+i+ = A T ji+=j ! j,j,q j q j _! i+i+ =i+>j A T _! i ii+ +^! i+iq j q j _! i+i+ = A T ji+=j _! j,q j q j i+i+_!=^ i+i+q js i+0;i+q j=i+>j A T i++^! i+i+q jz i+i+ _q i+q j^ A Tj, + j_! j,j,q jq j~! i+i+ + ^! i+i+ si0;i+ i is i i;i+q j q js i+0;i+ = A T ji+=j sj,0;j,q j q + j,jj,s j,j,;jM i+c i+q jF i+c i+= i+ i+q jJ i+c i+,0;i+q j= m i+r i+q jz j j _qj~! i+i+q j i+ Ti+J i+cq i+ j!TA.6 Reverse Partials with Initial Conditionsf i iq j=9i+F i c iq j+f i iq j= 69i+F i c iq jii =q j9i+]F iri c ii;i+q jXi+ Ai+f i+i+q + f iAi+ ij q j+^M i c iq j+A i+ i+i+q j+ g sii;i+A i+q jf i+ii =q j69i+]F iri c ii;i+ +^M i c iq j q j iq j= i iq j z i iReferencesXi+Ai+q j i+i+ +i+ + g sii;i+ A i+f i+i+q j[1] BALAFOUTIS,C.A.,AND PATEL,R.V.Dynamic Analysis **of**Robot Manipulators: A Cartesian Tensor Approach. KluwerAcademic Publishers, 1991.[2] BRUDERLIN, A.,AND WILLIAMS, L. **Motion** signal processing.In Computer Graphics (Aug. 1995), pp. 97–104. Proceedings**of** SIGGRAPH 95.[3] BURDETT,R.G.,SKRINAR,G.S.,AND SIMON,S.R.Comparison**of** mechanical work and metabolic energy consumptionduring normal gait. Journal **of** Orhopaedic Research 1,1(1983), 63–72.[4] COHEN, M. F. Interactive spacetime control for animation. InComputer Graphics (July 1992), pp. 293–302. Proceedings **of**SIGGRAPH 92.[5] GILL, P.E.,MURRAY,W.,AND WRIGHT,M.H. PracticalOptimization. Academic Press, 1981.

[6] HOLLERBACH, J. M. A recursive lagrangian formulation **of**manipulator dynamics and a comparative study **of** dynamicsformulation complexity. IEEE Transactions on Systems, Man,and Cybernetics SMC-10, 11 (Nov. 1980).[7] LIU, Z.,AND COHEN, M. F. An efficient symbolic interfaceto constraint based animation systems. In Proceedings **of**the 5th EuroGraphicsWorkshop on Animation and Simulation(Sept. 1994).[8] LIU, Z.,GORTLER, S.J.,AND COHEN, M. F. Hierarchicalspacetime control. In Computer Graphics (July 1994), pp. 35–42. Proceedings **of** SIGGRAPH 94.[9] UNUMA,M.,ANJYO,K.,AND TEKEUCHI, R. Fourier principlesfor emotion-based human figure animation. In ComputerGraphics (Aug. 1995), pp. 91–96. Proceedings **of** SIG-GRAPH 95.[10] WITKIN,A.,AND KASS, M. **Spacetime** constraints. In ComputerGraphics (Aug. 1988), pp. 159–168. Proceedings **of**SIGGRAPH 88.[11] WITKIN, A.,AND POPOVÍC, Z. **Motion** warping. In ComputerGraphics (Aug. 1995), pp. 105–108. Proceedings **of**SIGGRAPH 95.[12] ZHAO,J.,AND BADLER, N. I. Inverse kinematics positioning**using** non-linear programming for highly articulated figures.ACM Transactions on Graphics 13, 4 (Oct. 1994), 313–336.