Animation - Artemis

AnimationToolswithin MPEG-4MPEG-4 4 3D GraphicsCGICrete, June 2004Marius PREDAARTEMIS Project UnitInstitut Nationaldes TélécommunicationsFRANCE

MP4 AnimationPrinciple Rigid motionContinuous update of the 3D objectregistration in the scene DeformationContinuous update of the 3D objectgeometryObject’s feature update in time

Animation Toolswithin MPEG- 4OutlineGeneric animation- interpolators- animators- remote- programmaticSpecific Animation- virtual characters

Animation Toolswithin MPEG- 4OutlineGeneric animation- interpolators- animators- remote- programmaticSpecific Animation- virtual characters

MP4 Generic animationInterpolatorsPiece-wise linear functions definedby a set of pairs time-valueStandardized by MPEG-4 4 for:- position- orientation- scale- coordinate in IFS- normals in IFSStraightforward animation based on key-frames

MP4 Generic animationInterpolatorsPositionInterpolator {key[0, 0.0066667,0.013333, 0.02,…0.99333, 1]keyValue[-1647.2 409.05 78.94,…-1263.7 388.93 63.428]}…..Large amount of data for high quality, smooth animation

MP4 Generic animationReducing data size, 1st method: interpolator compressionTwo methods supported byMPEG-4OriginalInterpolatorPath preserving modeKey preserving modeInterpolatorAnalyzerThree schemes supported byMPEG-4HeaderEncoderKey ValueEncoderKey EncoderOrientation InterpolatorsCoordinate InterpolatorsPosition InterpolatorsEncodedBitstreamExploit temporal redundancy for the most common interpolators

MP4 Generic animationReducing data size, 1st method: interpolator compressionTwo methods supported byMPEG-4Path preserving modeKey preserving modeKey valueKey(time)N key valuesKey valueKey(time)N-1 key valuesThree schemes supported byMPEG-4Orientation InterpolatorsCoordinate InterpolatorsPosition InterpolatorsKey valueKey(time)N-2 key valuesKey valueKey(time)N-3 key valuesSub-sampling or re-sampling based on minimal distortion

MP4 Generic animationReducing data size, 1st method: interpolator compressionAt least 50:1w.r.t. VRMLAt least 30% betterthan Advanced BIFS CodingOriginalVRML50:1CompressionInterpolatorCompressionSameBitrateAdvancedBIFS CodingA dedicated elementary stream for IC, multiplexed into an AFX stream

MP4 Generic animationReducing data size, 2nd method: animators Path: cubic Bézier, , quadraticBézier, , NURBS Velocity control Cumulative animations Time SharingoffsetStandardized by MPEG-4 4 for:- position coupled with orientation- scaleComplex trajectories and time controlAnimation valuesAnimator 1fromTo 0.2 0.4Animator 2fromTo 0.8 10.2 0.4 0.8 1t

MP4 Generic animationRemote animation and Programmatic animation Remote animationBIFS-Anim& BIFS CommandBifs-AnimBifs-Commandscene ECMA Scriptpart of the scene description Java Codestandardized API for accessingthe scene graphScript NodeMPEG-J StreamscenesceneComplex application scenario can be build

Animation Toolswithin MPEG- 4OutlineGeneric animation- interpolators- animators- remote- programmaticSpecific Animation- virtual characters

MP4 VC animationWhat kind of VCs? Face, MP4 V1,1999 Body, MP4 Amd1,2000 Skinned Model, MP4 Part 16,2003Two frameworks: human-like (FBA) and generic skeleton (BBA)

MP4 VC animationHow integrate VCs into an MP4 scene? Nodes in the scene graph forstatic representation:FBA: Face, Body nodesBBA: SBSkinnedModel node Elementary Streams for theanimationFBA streamBBA streamStreaming ready : definition and animation in separate streams

MP4 VC animationFBA, Shape Modeling for Face Geometry:Polygonal mesh obtained byIndexed Face Set Texture:Bitmap image mapped on 3DobjectsCollection of 3D objects,

MP4 VC animationFBA, Motion parameterization for Face Face control points:84 standardized points Animation parameters - FAP:2 High Level- expressions (14)- visemes (6)66 Low level- control points translation- eyeball & head rotationFAP #59 "raise_l_cornerlip_o"FAP #53 "stretch_l_cornerlip_o"Target configurations + local control in standardized points

MP4 VC animationFBA, Deformation behavior for Face Region deformation induced bymotion parameters in theneighborhood of the control points Region rigid transforms inducedby motion parameters in theneighborhood of the control points(e.g. eyeball rotation)Deformation effect in the control point’s neighborhood

MP4 VC animationFBA, Shape Modeling for Body Geometry:Polygonal mesh for eachanatomical segment obtained byIndexed Face Set Texture:Bitmap image mapped on 3Dobjects Hierarchy:Parent-child relationshipCollection of hierarchical 3D objects

MP4 VC animationFBA, Motion parameterization for Body Animation Parameters - BAP:3 translations183 rotations110 user defined parameters Attached to the anatomicalsegments- standardized rotation axes

MP4 VC animationFBA, Motion parameterization for Body Animation Parameters - BAP:3 translations183 rotations110 user defined parameters Attached to the anatomicalsegments- standardized rotation planeRigid transform of anatomical segments, standardized directions

MP4 VC animationFBA, Deformation behavior for Body Rigid animation -> > joint seams∆jk Correction by local deformationijjBAP kiji∆ik

MP4 VC animationFBA, Deformation behavior for Body Rigid animation -> > joint seams Correction by local deformationLarge set of tables needed for realistic motion

MP4 VC animationFBA, Animation examplesSolder avatar by Michael Miller’s Animation parametersinvariance wrt bodydefinitionNancy avatar byCindy Reed-Ballreich’s... 1 0 0 1 1 0 0 1 ...the same BAP stream Composition with videoFast animation, easy to set up, correct but not impressive results

MP4 VC animationBBA, Shape Modeling for Generic skinned model Geometry:Seamless mesh: shapes sharingthe same vertices list Texture:Image Mapping on verticessub-setset Hierarchy:Skeleton layerMuscle layerCourtesy of VRLab, EPFLSeamless mesh affected by a hierarchical skeleton

MP4 VC animationBBA, Deformation controller Deformer definition:( Ci) = ( Si, V ( Si),µi)ii Deformer influence in anarbitrary vertex v:ϕi( ν ) = µ ( ν ) ω [ T ( ξ ) −ξ]iξk∑∈ψ( ν )ikikkA geometric virtual entity, of variable form, with influence on the skin

MP4 VC animationBBA, Deformation controller 1D controllers: bone & muscle for each bone and each muscle- a list of affected vertices- a measure of affectednessare providedRight balance between control parameters and influence volume

MP4 VC animationBBA, Motion parameterizationR aS a Complete transformation foreach bone- translation- rotation (EA, Q)- scaleT aB 3 NURBS control for muscle- control points- weights- knotB 1B 2B 4B 5Rigid transform of the bones, smooth deformation of the musclesB 0

MP4 VC animationBBA, set-up the VC Build the skeleton as ahierarchy of bones Attach the muscles to theskeleton Build the skin and attach it tothe bones and musclesMulti-layer layer structure

MP4 VC animationBBA, animate the VC Transform the bonesT a

MP4 VC animationBBA, animate the VC Transform the bonesT a

MP4 VC animationBBA, animate the VC Transform the bonesT a Deform the musclesN b

MP4 VC animationBBA, animate the VC Transform the bonesT a Deform the musclesN bDeformers motion propagated as skin deformation

MP4 VC animationBBA, animation example Able to build and animate anykind of skeleton Realistic and high qualityanimation Widely supported by ATsCourtesy of Televirtualxxx

MP4 VC animationAnimation stream compression: frame-based and DCT-basedUncompressedBAP/FAP/BBAFrame PFrame #n#UncompressedBAP/FAP/BBADCTSegment #n#PredictionQuantizationFrame IQuantizationSegmentPPredictionDC Coeff.SegmentIAC Coeff.ArithmeticcodingArithmeticcodingDC QHuffman codingDC QAC QHuffman codingBinary fileBinary fileBasic comp.: temporal prediction, quantization and entropy encoding

MP4 VC animationCompression performances Face, MP4 V1,2kbps Body, MP4 Amd1,5-30 kbps Skinned Model, MP4 Part 16,5-30 kbps for a human likeskeletonVery low bit-rate

MP4 VC animationMorph space, AFX Amendment 1 - 2005 Defined as a base mesh anda collection of target meshes Animation obtained byupdating the weights of thetarget meshes BBA stream updated toinclude morph data Usable for any kind of 3DobjectLocal and precise control for shape deformation

Thank you!Marius.Preda@int-evry.frevry.frARTEMIS Project UnitInstitut Nationaldes TélécommunicationsFRANCE