recent advances in haptic rendering and applications - Computer ...

RECENT ADVANCES INHAPTIC RENDERING ANDAPPLICATIONSEdited by Ming C. Lin and Miguel A. OtaduyCopyright © 20051

ORGANIZERSMing C. LinDepartment of Computer ScienceUniversity of North CarolinaChapel Hill, NC 27599-3175lin@cs.unc.eduPHO: 919-962-1974FAX: 919-962-1799Miguel OtaduyDepartment of Computer ScienceInstitute of Scientific ComputingETH ZentrumCH - 8092 Zürichotaduy@inf.ethz.ch3

LECTURERS• Elaine Cohen, University of Utahcohen@cs.utah.edu• David Johnson, University of Utahdejohnso@cs.utah.edu• Roberta Klatzky, Carnegie Mellon Universityklatzky@cmu.edu• Ming Lin, University of North Carolina at Chapel Hilllin@cs.unc.edu• Bill McNeely, Boeing Researchbill.mcneely@boeing.com• Miguel Otaduy, ETH Zurichotaduy@inf.ethz.ch• Dinesh Pai, Rutgers Universitydpai@cs.rutgers.edu• Ken Salisbury, Stanford University & Intuitive Surgery, Incjks@robotics.stanford.edu• Hong Tan, Purdue Universityhongtan@purdue.edu• Russell Taylor, University of North Carolina at Chapel Hilltaylorr@cs.unc.edu4

BIOGRAPHICAL SKETCHES OF LECTURERSElaine Cohen is a professor of computer science at the University of Utah. Sheis co-head of the Geometric Design and Computation Project and co-author ofGeometric Modeling with Splines: An Introduction (A. K. Peters, 2001). Dr.Cohen has focused her research in computer graphics, geometric modeling, andmanufacturing, with emphasis on complex sculptured models represented usingNURBS (Non-Uniform Rational B-splines) and NURBS-features. Results inmanufacturing research have been focused on automating process planning,automatic toolpath generation for models having many surfaces, optimizing bothwithin and across manufacturing stages and fixture automation. She has alsobeen working on issues related to telepresence and design collaborations invirtual environments. Recent research has produced algorithms for determiningboth visibility and accessibility of one object by another. Computation of suchinformation is necessary for manufacturing, assembly planning, graphics, andvirtual environments. Research in haptics has been focused on developing newapproaches to solving geometric computations such as fast and accurate contactand tracking algorithms for sculptured models and while research in hapticssystems has focused on realistic force feedback in distributed haptic systems forcomplex mechanical models. Dr. Cohen was the 2001 recipient of the Universityof Utah Distinguished Research Award and is a member of the ComputerScience and Telecommunications Board of the National Academies.Website: http://www.cs.utah.edu/~cohen/David Johnson is a research scientist at the University of Utah, School ofComputing, where he also received his PhD. His primary research interest is indistance algorithms for haptic rendering, motivated by the needs of virtualprototyping applications. He has given invited talks at Ford Motor Company, IntelResearch, and Institute for Mathematics and its Applications on these topics. Inaddition, he was a speaker at a Solid Modeling 2001 tutorial on haptic rendering.Website: http://www.cs.utah.edu/~dejohnso/Roberta Klatzky is a Professor of Psychology at Carnegie Mellon University,where she also holds faculty appointments in the Center for the Neural Basis ofCognition and the Human-Computer Interaction Institute. She served as Head ofPsychology from 1993 - 2003. She received a B.S. in mathematics from theUniversity of Michigan and a Ph.D. in experimental psychology from StanfordUniversity. Klatzky's research interests are in human perception and cognition,with special emphasis on haptic perception and spatial cognition. She has doneextensive research on haptic and visual object recognition, human navigation5

Bill McNeely obtained a Ph.D. in physics from Caltech in 1971 and for the next 6years pursued physics research in Hamburg, Germany. In 1977 he beganworking for Boeing Computer Services in Seattle, Washington, developingadvanced computer graphics for engineering applications. From 1981 to 1988 heserved as chief engineer at startup company TriVector Inc., creating softwareproducts for technical illustration. In 1988 he founded the software consultingcompany McNeely & Associates. In 1989 he returned to Boeing, where he nowholds the position of Technical Fellow. At Boeing he has pursued research inhigh-performance computer graphics, haptic simulation, and informationassurance.Website: http://www.boeing.com/phantom/Miguel Otaduy received his PhD in Computer Science in 2004 from theUniversity of North Carolina, Chapel Hill, supported by fellowships from theGovernment of the Basque Country and UNC Computer Science Alumni. Hereceived a bachelor’s degree in electrical engineering from University ofMondragon (Spain) in 2000. Between 1995 and 2000, he was a researchassistant at the research lab Ikerlan. In the summer of 2003 he worked atImmersion Medical. His research areas include haptic rendering, physicallybasedsimulation, collision detection, and geometric modeling. He and Linintroduced the novel concept of sensation preserving simplification for hapticrendering in their SIGGRAPH 2003 paper and proposed the first 6-DOF haptictexture rendering algorithm. He is currently a research associate at ETH Zurich.Website: http://graphics.ethz.ch/~otmiguel/Dinesh Pai is a Professor in the Department of Computer Science at RutgersUniversity. Previously, he was a Professor at the University of British Columbiaand a fellow of the BC Advanced Systems Institute. He received his Ph.D. fromCornell University. His research spans the areas of graphics, robotics, andhuman-computer interaction. His current work is in Human Simulation andMultisensory Computation; the latter includes multisensory simulation (integratinggraphics, haptics, and auditory displays) and multisensory modeling (based onmeasurement of shape, motion, reflectance, sounds, and contact forces). He isthe author of over 85 refereed publications, including 25 journal papers and 4SIGGRAPH papers. He has served on 6 editorial boards and on the programcommittees of all major conferences in Robotics and Graphics, and is a programchair for the 2004 Symposium on Computer Animation. He organized theSIGGRAPH2001 Panel “Newton's Nightmare: Reality meets Faux Physics,” andco-taught the SIGGRAPH 2003 course on “Physics-Based Sound Synthesis forGraphics and Interactive Systems.”Website: http://www.cs.rutgers.edu/~dpai/7

Russell Taylor is a Research Associate Professor of Computer Science,Physics & Astronomy, and Materials Science at the University of North Carolinaat Chapel Hill. He has spent his 10-year career building scientific visualizationsand interactive computer-graphics applications to help scientists betterunderstand their instrumentation and data. He was the Principal Investigator onUNC's NIH National Research Resource on Computer Graphics for MolecularStudies and Microscopy and is now Co-director of UNC's NanoScale ScienceResearch Group (www.cs.unc.edu/Research/nano). For the past two years,Russ has taught a "Visualization in the Sciences" course to computer science,physics, chemistry, statistics, and materials science graduate students. Thecourse is based in large part on the textbook “Information Visualization:Perception for Design” (www.cs.unc.edu/Courses/comp290-069). Dr. Taylor hasbeen a contributor to several past SIGGRAPH courses.Website: http://www.cs.unc.edu/~taylorr/9

COURSE SYLLABUSThis course is designed to cover the psychophysics, design guideline, andfundamental haptic rendering algorithms, e.g. 3 degree-of-freedom (DOF)force-only display, 6-DOF force-and-torque display, or vibrotactile displayfor wearable haptics, and their applications in virtual prototyping, medicalprocedures, scientific visualization, 3D modeling & CAD/CAM, digitalsculpting and other creative processes. We have assembled an excellentteam of researchers and developers from both academia and industry tocover topics on fundamental algorithm design and novel applications.FUNDAMENTALS IN HAPTIC RENDERING• Haptic Perception & Design Guidelines (Roberta Klatzky)• Haptic Display of Sculptured Surfaces and Models(Elaine Cohen & David Johnson)• 6-DOF Haptics (Voxel-sampling: Bill McNeely,Multiresolution: Miguel Otaduy,Spatialized Normal Cone: David Johnson)• Texture Rendering (Perceptual parameters: Hong Tan andForce model: Miguel Otaduy)• Modeling of Deformable Objects (Dinesh Pai)• Wearable Haptics (Hong Tan)APPLICATIONS• Virtual Prototyping (Bill McNeely)• Medical Applications (Kenneth Salisbury)• Scientific Visualization (Russell Taylor)• CAD/CAM & Model Design (Elaine Cohen)• Haptic Painting & Digital Sculpting (Ming Lin)• Reality-Based Modeling for Multimodal Display (Dinesh Pai)11

PRE-REQUISITESThis course is for programmers and researchers who have done some implementation of3D graphics and want to learn more about how to incorporate recent advances in hapticrendering with their 3D graphics applications or virtual environments. Familiarity withbasic 3D graphics, geometric operation and elementary physics is highly recommended.INTENDED AUDIENCEResearchers who have background in computer graphics and want to learn how to addhaptic interaction to simulated environments and those who are working in VR andvarious applications ranging from digital sculpting, medical training, scientificvisualization, CAD/CAM, rapid prototyping, engineering design, education and training,painting, digital sculpting, etc.12

COURSE SCHEDULE8:30amIntroduction and Overview (Ming Lin & Miguel Otaduy)1. Definition of some very basic terminology2. Brief overview of a graphics+haptics system (HW/SW/control)3. Roadmap for the course4. Introduction of the speakersSESSION I: DESIGN GUIDELINES AND BASIC POINT-BASEDTECHNIQUES8:45am9:30amHaptic Perception & Design Guidelines (Roberta Klatzky)1. Sensory aspects of haptics1.1 mechanoreceptor function1.2 other receptors: thermal, pain1.3 cutaneous vs. kinesthetic components of haptic sense2. Psychophysical aspects of haptics2.1 haptic features2.2 link between exploration and haptic properties3. Complementary functions of haptics and vision3.1 material vs. geometric properties3.2 differential accessibility of these properties3.3 haptic/visual integration4. Issues for design4.1 force feedback vs. array feedback4.2 need for psychophysical input in developing & evaluatinghaptic feedback devices4.3 challenges for technologyBasics of 3-DOF Haptic Display (Ken Salisbury & Ming Lin)1. Geometric representations: point-based, polygonal, NURBS, etc.2. Force models: penalty-based, god-objects, virtual proxy, and so on3. Fast proximity queries and collision response, e.g. H-Collide4. Friction, texture rendering, force shading, etc.5. Multi-threading for haptic displaySESSION II: 6-DOF HAPTIC RENDERING FOR OBJECT-OBJECTINTERACTION13

10:00am10:15amIntroduction to 6-DOF Haptic Display (Bill McNeely)Brief introduction to 6-DOF haptic rendering & issuesBREAK10:30am10:55am11:20am6-DOF Haptic Display using Voxel Sampling (Bill McNeely)& Applications to Virtual Prototyping1. The Voxmap PointShell (VPS) approaches1.1 overview1.2 comparison with other approaches1.3 enhancements: distance fields, geometric awareness,temporal coherence2. Considerations in large-scale haptic simulation2.1 importance of minimum 10Hz graphics frame rate2.2 dynamic pre-fetching of voxel data3. Applications to virtual prototyping3.1 haptic-enabled FlyThru3.2 Spaceball quasi-haptic interfaceSensation Preserving Simplification for 6-DOF Haptic Display(Miguel Otaduy)1. Needs for multiresolution approaches in 6-DOF haptic renderingof complex interactions2. Contact Levels of Detail (C-LOD)2.1 definition of C-LOD2.2 creation of simplification hierarchy3. Collision detection and C-LOD selection3.1 definition of error metrics3.2 on-the-fly LOD selection3.3 accelerated collision queries6-DOF Haptic Display of Sculptured Surfaces (David Johnson)1. Need for multiple point tracking2. Normal cones to solve collinearity condition, pt-surface, surface-surface3. Hybrid systems with local updatesSESSION III: HAPTIC RENDERING OF HIGHER-ORDER PRIMITIVES11:45am3-DOF Haptic Display of Sculptured Surfaces (Elaine Cohen)1. Introduce sculptured models, NURBS background, trimmedmodels2. Equations of distance, extremal distance, collinearity condition14

3. Local tracking, wall models, etc for sculptured models4. Tracking on trimmed NURBs models, in particular CAD/CAMmodels12:15pm12:15amQ & A (All Morning Speakers)LUNCHSESSION IV: RENDERING OF TEXTURES AND DEFORMABLESURFACES1:45pm2:00pm2:20pm2:45pmWearable Vibrotactile Haptic Displays (Hong Tan)Brief introduction of wearable haptic displays originally developedfor sensory substitution; discussion on the types of information thatcan be successfully conveyed by array-based wearable vibrotactiledisplays.Toward Realistic Haptic Rendering of Textures (Hong Tan)Recent work on assessing the perceptual quality of hapticallyrendered surface textures. Emphasis will be placed on a quantitativeparameter space for realistic haptic texture rendering, and on thetypes of perceptual instabilities commonly encountered and theirsources.Haptic Rendering of Textured Surfaces (Miguel Otaduy)1. Overview of 3-DoF haptic texture rendering methods.2. A force model for 6-DoF haptic texture rendering3. GPU-based penetration depth computationModeling of Deformable Objects (Dinesh Pai)Force rendering of deformations – Basics of Elasticity; Numericalmethods for BVPs (FEM,BEM); Precomputed Green's functions;Capacitance Matrix Algorithms; Cosserat models.SESSION V: NOVEL APPLICATIONS3:10pmReality-based Modeling for Multimodal Display (Dinesh Pai)1. Estimation theory; Contact force measurements; Visualmeasurements of deformation; Sound measurements;2. Case Study A: automated measurement with ACME;3. Case Study B: interactive measurement with HAVEN.15

3:30pm3:45pm4:20pm4:50pm5:15pmBREAKHaptics and Medicine (Kenneth Salisbury)1. Tissue Modeling2. Topological changes on deformable models: cutting, suturing, etc.3. Haptic interaction methods4. Simulation-based training, skills assessment and planning, etc.Applications in Scientific Visualization (Russell Taylor)1. Benefits of haptics for scientific visualization2. Haptic display of force fields2.1 for training Physics students2.2 for molecular docking2.3 for comprehending flows3. Haptic display of simulated models3.1 for molecular dynamics3.2 for electronics diagnosis training3.3 for medical training4. Haptic display of data:4.1 remote micromaching.4.2 display multiple data sets?5. Haptic control of instrumentation, e.g. scanned-probe microscopesPhysically-based Haptic Painting & Interaction with FluidMedia (Ming Lin)1. Modeling of 3D deformable virtual brushes & viscous paintmedia2. Haptic rendering of brush-canvas interaction3. Haptic display of interaction with fluid media (e.g. oil paint)Q & A and Conclusion (All Speakers)16

TABLE OF CONTENTSPREFACEPART I – Tutorial/Survey Chapters1. Introduction to Haptic Rendering ………………………………….A3by Miguel A. Otaduy and Ming C. Lin2. A Framework for Fast and Accurate CollisionDetection for Haptic Interaction ...……………………………...…A34by Arthur Gregory, Ming C. Lin, Stefan Gottschalk, and Russell Taylor3. Six Degree-of-Freedom Haptic RenderingUsing Voxel Sampling ……………………………………………...A42by William A. McNeely, Kevin D. Puterbaugh, and James J. Troy4. Advances in Voxel-Based 6-DOF Haptic Rendering …………………A50by William A. McNeely, Kevin D. Puterbaugh, and James J. Troy5. Sensation Preserving Simplification for Haptic Rendering ...…...A72by Miguel A. Otaduy and Ming C. Lin6. A Haptic System for Virtual Prototypingof Polygonal Models ..........................................................................A84by David E. Johnson, Peter Willemsen, and Elaine Cohen7. Direct Haptic Rendering of ComplexTrimmed NURBS Models ……………………………………………...A89by Thomas V. Thompson II and Elaine Cohen8. Haptic Rendering of Surface-to-SurfaceSculpted Model Interaction ………………………………………..A97by Donald D. Nelson, David E. Johnson, and Elaine Cohen9. Tactual Displays for Sensory Substitutionand Wearable Computers ………………………………………….A105by Hong Z. Tan and Alex Pentland10. Toward Realistic Haptic Rendering of Surface Textures ...…….A125by Seungmoon Choi and Hong Z. Tan17

11. Haptic Display of Interaction between Textured Models ………A133by Miguel A. Otaduy, Nitin Jain, Avneesh Sud, and Ming C. Lin12. A Unified Treatment of Elastostatic ContactSimulation for Real Time Haptics ……………………………….A141by Doug L. James and Dinesh K. Pai13. Scanning Physical Interaction Behavior of 3D Objects ………...A154by Dinesh K. Pai, Kees van cen Doel, Doug L. James, Jochen Lang, JohnE. Lloyd, Joshua L. Richmond, Som H. Yau14. The AHI: An Audio and Haptic Interfacefor Contact Interactions …………………………………………..A164by Derek DiFilippo and Dinesh K. Pai15. Haptics for Scientific Visualization ……………………………...A174by Russell M. Taylor II16. DAB: Interactive Haptic Paintingwith 3D Virtual Brushes ….............................................................A180by Bill Baxter, Vincent Scheib, Ming C. Lin, and Dinesh Manocha17. ArtNova: Touch-Enabled 3D Model Design ……………………..A188by Mark Foskey, Miguel A. Otaduy, and Ming C. Lin18

PART II – Presentation Slides1. Haptics: Introduction and Overview ……………………………….B3by Ming C. Lin and Miguel A. Otaduy2. Haptic Perception and Implications for Design ……………………B5by Roberta Klatzky3. Introduction to 3-DoF Haptic Rendering …………………………B12by Ming C. Lin4. Introduction to 6-DoF Haptic Display …………………………….B18by Bill McNeely5. Voxel Sampling for Six-DoF Haptic Rendering ………………….B19by Bill McNeely6. Sensation Preserving Simplificationfor 6-DoF Haptic Display …………………………………………..B24by Miguel A. Otaduy7. Haptic Rendering of Polygonal ModelsUsing Local Minimum Distances ………………………………….B37by David E. Johnson (Joint work with Elaine Cohenand Pete Willemsen)8. Haptic Rendering of Sculptured Models ………………………….B45by Elaine Cohen (Joint work with Tom Thompson,Don Nelson and David Johnson)9. Wearable Vibrotactile Displays ……………………………………B49by Hong Z. Tan10. Towards Realistic Haptic Rendering of Surface Texture ………..B52by Seungmoon Choi and Hong Z. Tan11. Haptic Rendering of Textured Surfaces …………………………..B56by Miguel A. Otaduy12. Modeling Deformable Objects for Haptics ……………………….B68by Dinesh K. Pai (Mostly joint work with Doug James)19

13. Reality-based Modeling for Hapticsand Multimodal Displays …………………………………………..B73by Dinesh K. Pai14. Applications in Scientific Visualization …………………………...B80by Russell M. Taylor II15. Haptic Interaction with Fluid Media ……………………………...B87by William Baxter and Ming C. Lin20

PREFACETo date, most human–computer interactive systems have focused primarily on thegraphical rendering of visual information and, to a lesser extent, on the display ofauditory information. Among all senses, the human haptic system provides unique andbidirectional communication between humans and their physical environment. Extendingthe frontier of visual computing, haptic interfaces, or force feedback devices, have thepotential to increase the quality of human-computer interaction by accommodating thesense of touch. They provide an attractive augmentation to visual display and enhancethe level of understanding of complex data sets. They have been effectively used for anumber of applications including molecular docking, manipulation of nano-materials,surgical training, virtual prototyping and digital sculpting.Compared with visual and auditory display, haptic rendering has extremely demandingcomputational requirements. In order to maintain a stable system while displayingsmooth and realistic forces and torques, haptic update rates of 1 KHz or more aretypically used. Haptics presents many new challenges to researchers and developers incomputer graphics and interactive techniques. Some of the critical issues include thedevelopment of novel data structures to encode shape and material properties, as well asnew techniques for data processing, information analysis, physical modeling, and hapticvisualization.This course will examine some of the latest developments on haptic rendering andapplications, while looking forward to exciting future research in this area. We willpresent novel haptic rendering algorithms and innovative applications that take advantageof haptic interaction sensory modality. Specifically we will discuss different renderingtechniques for various geometric representations (e.g. point-based, volumetric, polygonal,multiresolution, NURBS, distance fields, etc) and physical properties (rigid bodies,deformable models, fluid medium, etc), as well as textured surfaces and full-bodyinteraction (e.g. wearable haptics). We will also show how psychophysics of touch canprovide the foundational design guidelines for developing perceptually driven forcemodels and discuss issues to consider in validating new rendering techniques andevaluating haptic interfaces.In addition, we will also look at different approaches to design touch-enabled interfacesfor various applications, ranging from medical training, model design and maintainabilityanalysis for virtual prototyping, scientific visualization, 3D painting and mesh editing, todata acquisition for multi-modal display. These recent advances indicate promisingpotentials that haptic interfaces together with interactive 3D graphics can offer a fasterand more natural way of interacting with virtual environments and complex datasets indiverse applications.Ming C. Lin and Miguel A. Otaduy21

RECENT ADVANCES IN HAPTIC RENDERINGAND APPLICATIONSSupplementary Course NotesPART I – Tutorial and Survey ChaptersMing C. LinUniversity of North Carolina at Chapel HillMiguel A. OtaduyETH ZurichA1

Introduction to Haptic RenderingMiguel A. OtaduyMing C. LinDepartment of Computer ScienceUniversity of North Carolina at Chapel Hill1 Why Haptic Rendering?For a long time, human beings have dreamed of a virtual world where it is possible to interact with syntheticentities as if they were real. To date, the advances in computer graphics allow us to see virtual objectsand avatars, to hear them, to move them, and to touch them. It has been shown that the ability to touchvirtual objects increases the sense of presence in virtual environments [Insko 2001].Haptic rendering offers important applicability in engineering and medical training tasks. In this chapterwe introduce the concept of haptic rendering, and we briefly describe some of the basic techniques andapplications. In the first section we define some terminology, discuss the evolution of the research in hapticrendering, and introduce practical applications.1.1 DefinitionsThe term haptic (from the Greek haptesthai, meaning “to touch”) is the adjective used to describe somethingrelating to or based on the sense of touch. Haptic is to touching as visual is to seeing and as auditory isto hearing [Fisher et al. 2004].As described by Klatzky and Lederman [Klatzky and Lederman 2003], touch is one of the main avenues ofsensation, and it can be divided into cutaneous, kinesthetic, and haptic systems, based on the underlyingneural inputs. The cutaneous system employs receptors embedded in the skin, while the kinestheticsystem employs receptors located in muscles, tendons, and joints. The haptic sensory system employsboth cutaneous and kinesthetic receptors, but it differs in the sense that it is associated with an activeprocedure. Touch becomes active when the sensory inputs are combined with controlled body motion. Forexample, cutaneous touch becomes active when we explore a surface or grasp an object, while kinesthetictouch becomes active when we manipulate an object and touch other objects with it.Haptic rendering is defined as the process of computing and generating forces in response to user interactionswith virtual objects [Salisbury et al. 1995]. Several haptic rendering algorithms consider the paradigmof touching virtual objects with a single contact point. Rendering algorithms that follow this descriptionare called 3-DoF haptic rendering algorithms, because a point in 3D has only three DoFs. Other hapticrendering algorithms deal with the problem of rendering the forces and torques arising from the interactionof two virtual objects. This problem is called 6-DoF haptic rendering, because the grasped object has sixDoFs (position and orientation in 3D), and the haptic feedback comprises 3D force and torque. When weeat with a fork, write with a pen, or open a lock with a key, we are moving an object in 3D, and we feelthe interaction with other objects. This is, in essence, 6-DoF object manipulation with force-and-torquefeedback. Fig. 1 shows an example of a user experiencing haptic rendering. When we manipulate an objectand touch other objects with it, we perceive cutaneous feedback as the result of grasping, and kinestheticfeedback as the result of contact between objects.A3

1 WHY HAPTIC RENDERING? 2Figure 1: Example of Haptic Rendering. A person manipulates a virtual jaw using a haptic device(shown on the right of the image), and the interaction between jaws is displayed both visually and haptically.1.2 From Telerobotics to Haptic RenderingIn 1965, Ivan Sutherland [Sutherland 1965] proposed a multimodal display that would incorporate hapticfeedback into the interaction with virtual worlds. Before that, haptic feedback had already been used mainlyin two applications: flight simulators and master-slave robotic teleoperation. The early teleoperator systemshad mechanical linkages between the master and the slave. But, in 1954, Goertz and Thompson [Goertzand Thompson 1954] developed an electrical servomechanism that received feedback signals from sensorsmounted on the slave and applied forces to the master, thus producing haptic feedback.From there, haptic interfaces evolved in multiple directions, but there were two major breakthroughs.The first breakthrough was the idea of substituting the slave robot by a simulated system, in whichforces were computed using physically based simulations. The GROPE project at the University of NorthCarolina at Chapel Hill [Brooks, Jr. et al. 1990], lasting from 1967 to 1990, was the first one to addressthe synthesis of force feedback from simulated interactions. In particular, the aim of the project was toperform real-time simulation of 3D molecular-docking forces. The second breakthrough was the adventof computer-based Cartesian control for teleoperator systems [Bejczy and Salisbury 1980], enabling aseparation of the kinematic configurations of the master and the slave. Later, Cartesian control wasapplied to the manipulation of simulated slave robots [Kim and Bejczy 1991].Those first haptic systems were able to simulate the interaction of simple virtual objects only. Perhaps thefirst project to target computation of forces in the interaction with objects with rich geometric informationwas Minsky’s Sandpaper [Minsky et al. 1990]. Minsky et al. developed a planar force feedback system thatallowed the exploration of textures. A few years after Minsky’s work, Zilles and Salisbury presented analgorithm for 3-DoF haptic rendering of polygonal models [Zilles and Salisbury 1995]. Almost in parallelwith Zilles and Salisbury’s work, Massie and Salisbury [Massie and Salisbury 1994] designed the PHANToM,a stylus-based haptic interface that was later commercialized and has become one of the most commonlyused force-feedback devices. But in the late ’90s, research in haptic rendering revived one of the problemsthat first inspired virtual force feedback: 6-DoF haptic rendering or, in other words, grasping of a virtualobject and synthesis of kinesthetic feedback of the interaction between this object and its environment.A4

1 WHY HAPTIC RENDERING? 3Research in the field of haptics in the last 35 years has covered many more areas than what we havesummarized here. [Burdea 1996] gives a general survey of the field of haptics and [McLaughlin et al. 2002]discuss current research topics.1.3 Haptic Rendering for Virtual ManipulationCertain professional activities, such as training for high-risk operations or pre-production prototype testing,can benefit greatly from simulated reproductions. The fidelity of the simulated reproductions depends,among other factors, on the similarity of the behaviors of real and virtual objects. In the real world,solid objects cannot interpenetrate. Contact forces can be interpreted mathematically as constraint forcesimposed by penetration constraints. However, unless penetration constraints are explicitly imposed, virtualobjects are free to penetrate each other in virtual environments. Indeed, one of the most disconcertingexperiences in virtual environments is to pass through virtual objects [Insko et al. 2001; Slater and Usoh1993]. Virtual environments require the simulation of non-penetrating rigid body dynamics, and thisproblem has been extensively explored in the robotics and computer graphics literature [Baraff 1992;Mirtich 1996].It has been shown that being able to touch physical replicas of virtual objects (a technique known aspassive haptics [Insko 2001]) increases the sense of presence in virtual environments. This conclusion canprobably be generalized to the case of synthetic cutaneous feedback of the interaction with virtual objects.As reported by Brooks et al. [Brooks, Jr. et al. 1990], kinesthetic feedback radically improved situationawareness in virtual 3D molecular docking. Kinesthetic feedback has proved to enhance task performancein applications such as telerobotic object assembly [Hill and Salisbury 1977], virtual object assembly [Ungeret al. 2002], and virtual molecular docking [Ouh-Young 1990]. In particular, task completion time is shorterwith kinesthetic feedback in docking operations but not in prepositioning operations.To summarize, haptic rendering is especially useful in particular examples of training for high-risk operationsor pre-production prototype testing activities that involve intensive object manipulation and interactionwith the environment. Such examples include minimally invasive or endoscopic surgery [Edmondet al. 1997; Hayward et al. 1998] and virtual prototyping for assembly and maintainability assessment [Mc-Neely et al. 1999; Chen 1999; Andriot 2002; Wan and McNeely 2003]. Force feedback becomes particularlyimportant and useful in situations with limited visual feedback.1.4 3-DoF and 6-DoF Haptic RenderingMuch of the existing work in haptic rendering has focused on 3-DoF haptic rendering [Zilles and Salisbury1995; Ruspini et al. 1997; Thompson et al. 1997; Gregory et al. 1999; Ho et al. 1999]. Given a virtual objectA and the 3D position of a point p governed by an input device, 3-DoF haptic rendering can be summarizedas finding a contact point p ′ constrained to the surface of A. The contact force will be computed as afunction of p and p ′ . In a dynamic setting, and assuming that A is a polyhedron with n triangles,the problem of finding p ′ has an O(n) worst-case complexity. Using spatial partitioning strategies andexploiting motion coherence, however, the complexity becomes O(1) in many practical situations [Gregoryet al. 1999].This reduced complexity has made 3-DoF haptic rendering an attractive solution for many applicationswith virtual haptic feedback, such as: sculpting and deformation [Dachille et al. 1999; Gregory et al. 2000a;McDonnell et al. 2001], painting [Johnson et al. 1999; Gregory et al. 2000a; Foskey et al. 2002], volumevisualization [Avila and Sobierajski 1996], nanomanipulation [Taylor et al. 1993], and training for diversesurgical operations [Kuhnapfel et al. 1997; Gibson et al. 1997]. In each of these applications, the interactionbetween the subject and the virtual objects is sufficiently captured by a point-surface contact model.In 6-DoF manipulation and exploration, however, when a subject grasps an object and touches otherobjects in the environment, the interaction generally cannot be modeled by a point-surface contact. Onereason is the existence of multiple contacts that impose multiple simultaneous non-penetration constraintsA5

2 THE CHALLENGES 4on the grasped object. In a simple 6-DoF manipulation example, such as the insertion of a peg in a hole,the grasped object (i.e., the peg) collides at multiple points with the rest of the scene (i.e., the walls ofthe hole and the surrounding surface). This contact configuration cannot be modeled as a point-objectcontact. Another reason is that the grasped object presents six DoFs, 3D translation and rotation, asopposed to the three DoFs of a point. The feasible trajectories of the peg are embedded in a 6-dimensionalspace with translational and rotational constraints, that cannot be captured with three DoFs.Note that some cases of object-object interaction have been modeled in practice by ray-surface contact[Basdogan et al. 1997]. In particular, several surgical procedures are performed with 4-DoF tools (e.g.,laparoscopy), and this constraint has been exploited in training simulators with haptic feedback [Çavuşoğluet al. 2002]. Nevertheless, these approximations are valid only in a limited number of situations and cannotcapture full 6-DoF object manipulation.2 The ChallengesHaptic rendering is in essence an interactive activity, and its realization is mostly handicapped by twoconflicting challenges: high required update rates and the computational cost. In this section we outlinethe computational pipeline of haptic rendering, and we discuss associated challenges.2.1 Haptic Rendering PipelineHaptic rendering comprises two main tasks. One of them is the computation of the position and/ororientation of the virtual probe grasped by the user. The other one is the computation of contact forceand/or torque that are fed back to the user. The existing methods for haptic rendering can be classifiedinto two large groups based on their overall pipelines.In direct rendering methods [Nelson et al. 1999; Gregory et al. 2000b; Kim et al. 2003; Johnson andWillemsen 2003; Johnson and Willemsen 2004], the position and/or orientation of the haptic device areapplied directly to the grasped probe. Collision detection is performed between the grasped probe and thevirtual objects, and collision response is applied to the grasped probe as a function of object separation orpenetration depth. The resulting contact force and/or torque are directly fed back to the user.In virtual coupling methods [Chang and Colgate 1997; Berkelman 1999; McNeely et al. 1999; Ruspiniand Khatib 2000; Wan and McNeely 2003], the position and/or orientation of the haptic device are setas goals for the grasped probe, and a virtual viscoelastic coupling [Colgate et al. 1995] produces a forcethat attracts the grasped probe to its goals. Collision detection and response are performed between thegrasped probe and the virtual objects. The coupling force and/or torque are combined with the collisionresponse in order to compute the position and/or orientation of the grasped probe. The same couplingforce and/or torque are fed back to the user.In Sec. 8, I describe the different existing methods for 6-DoF haptic rendering in more detail, and Idiscuss their advantages and disadvantages. Also, as explained in more detail in Sec. 4, there are twomajor types of haptic devices, and for each type of device the rendering pipeline presents slight variations.Impedance-type devices read the position and orientation of the handle of the device and control the forceand torque applied to the user. Admittance-type devices read the force and torque applied by the userand control the position and orientation of the handle of the device.2.2 Force Update RateThe ultimate goal of haptic rendering is to provide force feedback to the user. This goal is achieved bycontrolling the handle of the haptic device, which is in fact the end-effector of a robotic manipulator. Whenthe user holds the handle, he or she experiences kinesthetic feedback. The entire haptic rendering systemA6

2 THE CHALLENGES 5is regarded as a mechanical impedance that sets a transformation between the position and velocity of thehandle of the device and the applied force.The quality of haptic rendering can be measured in terms of the dynamic range of impedances that canbe simulated in a stable manner [Colgate and Brown 1994]. When the user moves the haptic device in freespace, the perceived impedance should be very low (i.e., small force), and when the grasped virtual objecttouches other rigid objects, the perceived impedance should be high (i.e., high stiffness and/or dampingof the constraint). The quality of haptic rendering can also be measured in terms of the responsiveness ofthe simulation [Brooks, Jr. et al. 1990; Berkelman 1999]. In free-space motion the grasped probe shouldrespond quickly to the motion of the user. Similarly, when the grasped probe collides with a virtual wall,the user should stop quickly, in response to the motion constraint.With impedance-type devices, virtual walls are implemented as large stiffness values in the simulation.In haptic rendering, the user is part of a closed-loop sampled dynamic system [Colgate and Schenkel 1994],along with the device and the virtual environment, and the existence of sampling and latency phenomenacan induce unstable behavior under large stiffness values. System instability is directly perceived by theuser in the form of disturbing oscillations. A key factor for achieving a high dynamic range of impedances(i.e., stiff virtual walls) while ensuring stable rendering is the computation of feedback forces at a highupdate rate [Colgate and Schenkel 1994; Colgate and Brown 1994]. Brooks et al. [Brooks, Jr. et al. 1990]reported that, in the rendering of textured surfaces, users were able to perceive performance differences atforce update rates between 500Hz and 1kHz.A more detailed description of the stability issues involved in the synthesis of force feedback, and adescription of related work, are given in Sec. 4. Although here we have focused on impedance-type hapticdevices, similar conclusions can be drawn for admittance-type devices (See [Adams and Hannaford 1998]and Sec. 4).2.3 Contact DeterminationThe computation of non-penetrating rigid-body dynamics of the grasped probe and, ultimately, synthesis ofhaptic feedback require a model of collision response. Forces between the virtual objects must be computedfrom contact information. Determining whether two virtual objects collide (i.e., intersect) is not enough,and additional information, such as penetration distance, contact points, contact normals, and so forth,need to be computed. Contact determination describes the operation of obtaining the contact informationnecessary for collision response [Baraff 1992].For two interacting virtual objects, collision response can be computed as a function of object separation,with worst-case cost O(mn), or penetration depth, with a complexity bound of Ω(m 3 n 3 ). But collisionresponse can also be applied at multiple contacts simultaneously. Given two objects A and B with m andn triangles respectively, contacts can be defined as pairs of intersecting triangles or pairs of triangles insidea distance tolerance. The number of pairs of intersecting triangles is O(mn) in worst-case pathologicalcases, and the number of pairs of triangles inside a tolerance can be O(mn) in practical cases. In Sec. 5,we discuss in more detail existing techniques for determining the contact information.The cost of contact determination depends largely on factors such as the convexity of the interactingobjects or the contact configuration. There is no direct connection between the polygonal complexityof the objects and the cost of contact determination but, as a reference, existing exact collision detectionmethods can barely execute contact queries for force feedback between pairs of objects with 1, 000 trianglesin complex contact scenarios [Kim et al. 2003] at force update rates of 1kHz.Contact determination becomes particularly expensive in the interaction between textured surfaces.Studies have been done on the highest texture resolution that can be perceived through cutaneous touch,but there are no clear results regarding the highest resolution that can be perceived kinesthetically throughan intermediate object. It is known that, in the latter case, texture-induced roughness perception is encodedin vibratory motion [Klatzky and Lederman 2002]. Psychophysics researchers report that 1mm texturesA7

3 PSYCHOPHYSICS OF HAPTICS 6are clearly perceivable, and perceived roughness appears to be even greater with finer textures [Ledermanet al. 2000]. Based on Shannon’s sampling theorem, a 10cm × 10cm plate with a sinusoidal texture of1mm in orthogonal directions is barely correctly sampled with 40, 000 vertices. This measure gives an ideaof the triangulation density required for capturing texture information of complex textured objects. Notethat the triangulation density may grow by orders of magnitude if the textures are not sinusoidal and/orif information about normals and curvatures is also needed.3 Psychophysics of HapticsIn the design of contact determination algorithms for haptic rendering, it is crucial to understand thepsychophysics of touch and to account for perceptual factors. The structure and behavior of humantouch have been studied extensively in the field of psychology. The topics analyzed by researchers includecharacterization of sensory phenomena as well as cognitive and memory processes.Haptic perception of physical properties includes a first step of stimulus registration and communicationto the thalamus, followed by a second step of higher-level processing. Perceptual measures can be originatedby individual mechanoreceptors but also by the integration of inputs from populations of differentsensory units [Klatzky and Lederman 2003]. Klatzky and Lederman [Klatzky and Lederman 2003] discussobject and surface properties that are perceived through the sense of touch (e.g., texture, hardness, andweight) and divide them between geometric and material properties. They also analyze active exploratoryprocedures (e.g., lateral motion, pressure, or unsupported holding) typically conducted by subjects in orderto capture information about the different properties.Knowing the exploratory procedure(s) associated with a particular object or surface property, researchershave studied the influence of various parameters on the accuracy and magnitude of sensory outputs.Perceptual studies on tactile feature detection and identification, as well as studies on texture or roughnessperception are of particular interest for haptic rendering. In this section we summarize existing researchon perception of surface features and perception of roughness, and then we discuss issues associated withthe interaction of visual and haptic modalities.3.1 Perception of Surface FeaturesKlatzky and Lederman describe two different exploratory procedures followed by subjects in order tocapture shape attributes and identify features and objects. In haptic glance [Klatzky and Lederman 1995],subjects extract information from a brief haptic exposure of the object surface. Then they perform higherlevelprocessing for determining the identity of the object or other attributes. In contour following [Klatzkyand Lederman 2003], subjects create a spatiotemporal map of surface attributes, such as curvature, thatserves as the pattern for feature identification. Contact determination algorithms attempt to describethe geometric interaction between virtual objects. The instantaneous nature of haptic glance [Klatzkyand Lederman 1995] makes it strongly dependent on purely geometric attributes, unlike the temporaldependency of contour following.Klatzky and Lederman [Klatzky and Lederman 1995] conducted experiments in which subjects wereinstructed to identify objects from brief cutaneous exposures (i.e., haptic glances). Subjects had an advancehypothesis of the nature of the object. The purpose of the study was to discover how, and how well, subjectsidentify objects from brief contact. According to Klatzky and Lederman, during haptic glance a subjecthas access to three pieces of information: roughness, compliance, and local features. Roughness andcompliance are material properties that can be extracted from lower-level processing, while local featurescan lead to object identification by feature matching during higher-level processing. In the experiments,highest identification accuracy was achieved with small objects, whose shapes fit on a fingertip. Klatzky andLederman concluded that large contact area helped in the identification of textures or patterns, although itwas better to have a stimulus of a size comparable to or just slightly smaller than that of the contact areaA8

3 PSYCHOPHYSICS OF HAPTICS 7for the identification of geometric surface features. The experiments conducted by Klatzky and Ledermanposit an interesting relation between feature size and contact area during cutaneous perception.Okamura and Cutkosky [Okamura and Cutkosky 1999; Okamura and Cutkosky 2001] analyzed featuredetection in robotic exploration, which can be regarded as a case of object-object interaction. Theycharacterized geometric surface features based on the ratios of their curvatures to the radii of the roboticfingertips acquiring the surface data. They observed that a larger fingertip, which provides a larger contactarea, can miss small geometric features. To summarize, the studies by Klatzky and Lederman [Klatzkyand Lederman 1995] and Okamura and Cutkosky [Okamura and Cutkosky 1999; Okamura and Cutkosky2001] lead to the observation that human haptic perception of the existence of a geometric surface featuredepends on the ratio between the contact area and the size of the feature, not the absolute size of thefeature itself. This observation has driven the design of multiresolution contact determination algorithmsfor haptic rendering [Otaduy and Lin 2003b].3.2 Perception of Texture and RoughnessKlatzky and Lederman [Klatzky and Lederman 2003] describe a textured surface as a surface with protuberantelements arising from a relatively homogeneous substrate. Interaction with a textured surfaceresults in perception of roughness. Existing research on the psychophysics of texture perception indicates aclear dichotomy of exploratory procedures: (a) perception of texture with the bare skin, and (b) perceptionthrough an intermediate (rigid) object, a probe.Most of the research efforts have been directed towards the characterization of cutaneous perceptionof textures. Katz [Katz 1989] suggested that roughness is perceived through a combination of spatialand vibratory codes during direct interaction with the skin. More recent evidence demonstrates thatstatic pressure distribution plays a dominant role in perception of coarse textures (features larger than1mm) [Lederman 1974; Connor and Johnson 1992], but motion-induced vibration is necessary for perceivingfine textures [LaMotte and Srinivasan 1991; Hollins and Risner 2000].As pointed out by Klatzky and Lederman [Klatzky and Lederman 2002], in object-object interactionroughness is encoded in vibratory motion transmitted to the subject. In the last few years, Klatzky andLederman have directed experiments that analyze the influence of several factors on roughness perceptionthrough a rigid probe. Klatzky et al. [Klatzky et al. 2003] distinguished three types of factors that may affectthe perceived magnitude of roughness: interobject physical interaction, skin- and limb-induced filteringprior to cutaneous and kinesthetic perception, and higher-level factors such as efferent commands. Thedesign of contact determination and collision response algorithms for haptic texture rendering is mostlyconcerned with factors related to the physical interaction between objects: object geometry [Ledermanet al. 2000; Klatzky et al. 2003], applied force [Lederman et al. 2000], and exploratory speed [Ledermanet al. 1999; Klatzky et al. 2003]. The influence of these factors has been addressed in the design of haptictexture rendering algorithms [Otaduy et al. 2004].The experiments conducted by Klatzky and Lederman to characterize roughness perception [Klatzky andLederman 2002] used a common setup: subjects explored a textured plate with a probe with a sphericaltip, and then they reported a subjective measure of roughness. Plates of jittered raised dots were used,and the mean frequency of dot distribution was one of the variables in the experiments. The resulting datawas analyzed by plotting subjective roughness values vs. dot interspacing in logarithmic graphs.Klatzky and Lederman [Klatzky and Lederman 1999] compared graphs of roughness vs. texture spacing(a) with finger exploration and (b) with a rigid probe. They concluded that, in the range of their data,roughness functions were best fit by linear approximations in finger exploration and by quadratic approximationsin probe-based exploration. In other words, when perceived through a rigid spherical probe,roughness initially increases as texture spacing increases, but, after reaching a maximum roughness value,it decreases again. Based on this finding, the influence of other factors on roughness perception can becharacterized by the maximum value of roughness and the value of texture spacing at which this maximumA9

4 STABILITY AND CONTROL THEORY APPLIED TO HAPTIC RENDERING 8takes place.Lederman et al. [Lederman et al. 2000] demonstrated that the diameter of the spherical probe playsa crucial role in the maximum value of perceived roughness and the location of the maximum. Theroughness peak is higher for smaller probes, and it occurs at smaller texture spacing values. Ledermanet al. [Lederman et al. 2000] also studied the influence of the applied normal force during exploration.Roughness is higher for larger force, but the influence on the location of the peak is negligible. The effectof exploratory speed was studied by Lederman et al. [Lederman et al. 1999]. They found that the peak ofroughness occurs at larger texture spacing for higher speed. Also, with higher speed, textured plates feelsmoother at small texture spacing, and rougher at large spacing values. The studies reflected that speedhas a stronger effect in passive interaction than in active interaction.3.3 Haptic and Visual Cross-modal InteractionHaptic rendering is often presented along with visual display. Therefore, it is important to understand theissues involved in cross-modal interaction. Klatzky and Lederman [Klatzky and Lederman 2003] discussaspects of visual and haptic cross-modal integration from two perspectives: attention and dominance.Spence et al. [Spence et al. 2000] have studied how visual and tactile cues can influence a subject’sattention. Their conclusions are that visual and tactile cues are treated together in a single attentionalmechanism, and wrong attention cues can affect perception negatively.Sensory dominance is usually studied by analyzing perceptual discrepancies in situations where crossmodalintegration yields a unitary perceptual response. One example of relevance for this dissertationis the detection of object collision. During object manipulation, humans determine whether two objectsare in contact based on a combination of visual and haptic cues. Early studies of sensory dominanceseemed to point to a strong dominance of visual cues over haptic cues [Rock and Victor 1964], but in thelast decades psychologists agree that sensory inputs are weighted based on their statistical reliability orrelative appropriateness, measured in terms of accuracy, precision, and cue availability [Heller et al. 1999;Ernst and Banks 2001; Klatzky and Lederman 2003].The design of contact determination algorithms can also benefit from existing studies on the visualperception of collisions in computer animations. O’Sullivan and her colleagues [O’Sullivan et al. 1999;O’Sullivan and Dingliana 2001; O’Sullivan et al. 2003] have investigated different factors affecting visualcollision perception, including eccentricity, separation, distractors, causality, and accuracy of simulationresults. Basing their work on a model of human visual perception validated by psychophysical experiments,they demonstrated the feasibility of using these factors for scheduling interruptible collision detection amonglarge numbers of visually homogeneous objects.4 Stability and Control Theory Applied to Haptic RenderingIn haptic rendering, the human user is part of the dynamic system, along with the haptic device andthe computer implementation of the virtual environment. The complete human-in-the-loop system canbe regarded as a sampled-data system [Colgate and Schenkel 1994], with a continuous component (theuser and the device) and a discrete one (the implementation of the virtual environment and the devicecontroller). Stability becomes a crucial feature, because instabilities in the system can produce oscillationsthat distort the perception of the virtual environment, or uncontrolled motion of the device that can evenhurt the user. In Sec. 2.2, we have briefly discussed the importance of stability for haptic rendering, andwe have introduced the effect of the force update rate on stability. In this section we review and discussin more detail existing work in control theory related to stability analysis of haptic rendering.A10

4 STABILITY AND CONTROL THEORY APPLIED TO HAPTIC RENDERING 94.1 Mechanical Impedance ControlThe concept of mechanical impedance extends the notion of electrical impedance and refers to the quotientbetween force and velocity. Hogan [Hogan 1985] introduced the idea of impedance control for contact tasksin manipulation. Earlier techniques controlled contact force, robot velocity, or both, but Hogan suggestedcontrolling directly the mechanical impedance, which governs the dynamic properties of the system. Whenthe end effector of a robot touches a rigid surface, it suffers a drastic change of mechanical impedance,from low impedance in free space, to high impedance during contact. This phenomenon imposes seriousdifficulties on earlier control techniques, inducing instabilities.The function of a haptic device is to display the feedback force of a virtual world to a human user. Hapticdevices present control challenges very similar to those of manipulators for contact tasks. As introducedin Sec. 2.1, there are two major ways of controlling a haptic device: impedance control and admittancecontrol. In impedance control, the user moves the device, and the controller produces a force dependenton the interaction in the virtual world. In admittance control, the user applies a force to the device, andthe controller moves the device according to the virtual interaction.In both impedance and admittance control, high control gains can induce instabilities. In impedancecontrol, instabilities may arise in the simulation of stiff virtual surfaces. The device must react withlarge changes in force to small changes in the position. Conversely, in admittance control, rendering astiff virtual surface is not a challenging problem, because it is implemented as a low controller gain. Inadmittance control, however, instabilities may arise during free-space motion in the virtual world, becausethe device must move at high velocities under small applied forces, or when the device rests on a stiffphysical surface. Impedance and admittance control can therefore be regarded as complementary controltechniques, best suited for opposite applications. Following the unifying framework presented by Adamsand Hannaford [Adams and Hannaford 1998], Contact determination and force computation algorithmsare often independent of the control strategy.4.2 Stable Rendering of Virtual WallsSince the introduction of impedance control by Hogan [Hogan 1985], the analysis of the stability of hapticdevices and haptic rendering algorithms has focused on the problem of rendering stiff virtual walls. Thiswas known to be a complex problem at early stages of research in haptic rendering [Kilpatrick 1976], butimpedance control simplified the analysis, because a virtual wall can be modeled easily using stiffness andviscosity parameters.Ouh-Young [Ouh-Young 1990] created a discrete model of the Argonne ARM and the human arm andanalyzed the influence of force update rate on the stability and responsiveness of the system. Minsky,Brooks, et al. [Minsky et al. 1990; Brooks, Jr. et al. 1990] observed that update rates as high as 500Hz or1kHz might be necessary in order to achieve stability.Colgate and Brown [Colgate and Brown 1994] coined the term Z-Width for describing the range ofmechanical impedances that a haptic device can render while guaranteeing stability. They concluded thatphysical dissipation is essential for achieving stability and that the maximum achievable virtual stiffnessis proportional to the update rate. They also analyzed the influence of position sensors and quantization,and concluded that sensor resolution must be maximized and the velocity signal must be filtered.Almost in parallel, Salcudean and Vlaar [Salcudean and Vlaar 1994] studied haptic rendering of virtualwalls, and techniques for improving the fidelity of the rendering. They compared a continuous model of avirtual wall with a discrete model that accounts for differentiation of the position signal. The continuousmodel is unconditionally stable, but this is not true for the discrete model. Moreover, in the discrete modelfast damping of contact oscillations is possible only with rather low contact stiffness and, as indicated byColgate and Brown too [Colgate and Brown 1994], this value of stiffness is proportional to the updaterate. Salcudean and Vlaar proposed the addition of braking pulses, proportional to collision velocity, forimproving the perception of virtual walls.A11

4 STABILITY AND CONTROL THEORY APPLIED TO HAPTIC RENDERING 104.3 Passivity and Virtual CouplingA subsystem is passive if it does not add energy to the global system. Passivity is a powerful tool foranalyzing stability of coupled systems, because the coupled system obtained from two passive subsystemsis always stable. Colgate and his colleagues were the first to apply passivity criteria to the analysis ofstability in haptic rendering of virtual walls [Colgate et al. 1993]. Passivity-based analysis has enabledseparate study of the behavior of the human subsystem, the haptic device, and the virtual environment inforce-feedback systems.4.3.1 Human Sensing and Control BandwidthsHogan discovered that the human neuromuscular system exhibits externally simple, springlike behavior[Hogan 1986]. This finding implies that the human arm holding a haptic device can be regarded as apassive subsystem, and the stability analysis can focus on the haptic device and the virtual environment.Note that human limbs are not passive in all conditions, but the bandwidth at which a subject canperform active motions is very low compared to the frequencies at which stability problems may arise.Some authors [Shimoga 1992; Burdea 1996] report that the bandwidth at which humans can performcontrolled actions with the hand or fingers is between 5 and 10Hz. On the other hand, sensing bandwidthcan be as high as 20 to 30Hz for proprioception, 400Hz for tactile sensing, and 5 to 10kHz for roughnessperception.4.3.2 Passivity of Virtual WallsColgate and Schenkel [Colgate and Schenkel 1994] observed that the oscillations perceived by a haptic userduring system instability are a result of active behavior of the force-feedback system. This active behavioris a consequence of time delay and loss of information inherent in sampled-data systems, as suggested byothers before [Brooks, Jr. et al. 1990]. Colgate and Schenkel formulated passivity conditions in hapticrendering of a virtual wall. For that analysis, they modeled the virtual wall as a viscoelastic unilateralconstraint, and they accounted for the continuous dynamics of the haptic device, sampling of the positionsignal, discrete differentiation for obtaining velocity, and a zero-order hold of the output force. Theyreached a sufficient condition for passivity that relates the stiffness K and damping B of the virtual wall,the inherent damping b of the device, and the sampling period T:b > KT2+ B. (1)4.3.3 Stability of Non-linear Virtual EnvironmentsAfter deriving stability conditions for rendering virtual walls modeled as unilateral linear constraints,Colgate and his colleagues considered more complex environments [Colgate et al. 1995]. A general virtualenvironment is non-linear, and it presents multiple and variable constraints. Their approach enforces adiscrete-time passive implementation of the virtual environment and sets a multidimensional viscoelasticvirtual coupling between the virtual environment and the haptic display. In this way, the stability of thesystem is guaranteed as long as the virtual coupling is itself passive, and this condition can be analyzed usingthe same techniques as those used for virtual walls [Colgate and Schenkel 1994]. As a result of Colgate’svirtual coupling [Colgate et al. 1995], the complexity of the problem was shifted towards designing a passivesolution of virtual world dynamics. As noted by Colgate et al. [Colgate et al. 1995], one possible way toenforce passivity in rigid body dynamics simulation is to use implicit integration with penalty methods.Adams and Hannaford [Adams and Hannaford 1998] provided a framework for analyzing stability withadmittance-type and impedance-type haptic devices. They derived stability conditions for coupled systemsbased on network theory. They also extended the concept of virtual coupling to admittance-type devices.A12

5 COLLISION DETECTION 11Miller et al. [Miller et al. 1990] extended Colgate’s passivity analysis techniques, relaxing the requirementof passive virtual environments but enforcing cyclo-passivity of the complete system. Hannaford and hiscolleagues [Hannaford et al. 2002] investigated the use of adaptive controllers instead of the traditionalfixed-value virtual couplings. They designed passivity observers and passivity controllers for dissipatingthe excess of energy generated by the virtual environment.4.4 Multirate Approximation TechniquesMultirate approximation techniques, though simple, have been successful in improving the stability andresponsiveness of haptic rendering systems. The idea is to perform a full update of the virtual environmentat a low frequency (limited by computational resources and the complexity of the system) and to use asimplified approximation for performing high-frequency updates of force feedback.Adachi [Adachi et al. 1995] proposed an intermediate representation for haptic display of complex polygonalobjects. In a slow collision detection thread, he computed a plane that served as a unilateral constraintin the force-feedback thread. This technique was later adapted by Mark et al. [Mark et al. 1996], whointerpolated the intermediate representation between updates. This approach enables higher stiffness valuesthan approaches that compute the feedback force values at the rate imposed by collision detection.More recently, a similar multirate approach has been followed by many authors for haptic interaction withdeformable models [Astley and Hayward 1998; Çavuşoğlu and Tendick 2000; Duriez et al. 2004]. Ellis etal. [Ellis et al. 1997] produce higher-quality rendering by upsampling directly the output force values.5 Collision DetectionCollision detection has received much attention in robotics, computational geometry, and computer graphics.Some researchers have investigated the problem of interference detection as a mechanism for indicatingwhether object configurations are valid or not. Others have tackled the problems of computing separationor penetration distances, with the objective of applying collision response in simulated environments. Theexisting work on collision detection can be classified based on the types of models handled: 2-manifoldpolyhedral models, polygon soups, curved surfaces, etc. In this section we focus on collision detection forpolyhedral models. The vast majority of the algorithms used in practice proceed in two steps: first theycull large portions of the objects that are not in close proximity, using spatial partitioning, hierarchicaltechniques, or visibility-based properties, and then they perform primitive-level tests.In this section, we first describe the problems of interference detection and computation of separationdistance between polyhedra, with an emphasis on algorithms specialized for convex polyhedra. Thenwe survey algorithms for the computation of penetration depth, the use of hierarchical techniques, andmultiresolution collision detection. we conclude the section by covering briefly the use of graphics processorsfor collision detection and the topic of continuous collision detection. For more information on collisiondetection, please refer to surveys on the topic [Lin and Gottschalk 1998; Klosowski et al. 1998; Lin andManocha 2004].5.1 Proximity Queries Between Convex PolyhedraThe property of convexity has been exploited in algorithms with sublinear cost for detecting interferenceor computing the closest distance between two polyhedra. Detecting whether two convex polyhedra intersectcan be posed as a linear programming problem, searching for the coefficients of a separating plane.Well-known linear programming algorithms [Seidel 1990] can run in expected linear time due to the lowdimensionality of the problem.The separation distance between two polyhedra A and B is equal to the distance from the origin tothe Minkowski sum of A and −B [Cameron and Culley 1986]. This property was exploited by GilbertA13

5 COLLISION DETECTION 12et al. [Gilbert et al. 1988] in order to design a convex optimization algorithm (known as GJK) forcomputing the separation distance between convex polyhedra, with linear-time performance in practice.Cameron [Cameron 1997] modified the GJK algorithm to exploit motion coherence in the initialization ofthe convex optimization at every frame for dynamic problems, achieving nearly constant running-time inpractice.Lin and Canny [Lin and Canny 1991; Lin 1993] designed an algorithm for computing separation distanceby tracking the closest features between convex polyhedra. Their algorithm “walks” on the surfaces of thepolyhedra until it finds two features that lie on each other’s Voronoi region. Exploiting motion coherenceand geometric locality, Voronoi marching runs in nearly constant time per frame. Mirtich [Mirtich 1998b]later improved the robustness of this algorithm.Given polyhedra A and B with m and n polygons respectively, Dobkin and Kirkpatrick [Dobkin andKirkpatrick 1990] proposed an algorithm for interference detection with O(log m log n) time complexitythat uses hierarchical representations of the polyhedra. Others have also exploited the use of hierarchicalconvex representations along with temporal coherence in order to accelerate queries in dynamic scenes.Guibas et al. [Guibas et al. 1999] employ the inner hierarchies suggested by Dobkin and Kirkpatrick,but they perform faster multilevel walking. Ehmann and Lin [Ehmann and Lin 2000] employ a modifiedversion of Dobkin and Kirkpatrick’s outer hierarchies, computed using simplification techniques, along witha multilevel implementation of Lin and Canny’s Voronoi marching [Lin and Canny 1991].5.2 Penetration DepthThe penetration depth between two intersecting polyhedra A and B is defined as the minimum translationaldistance required for separating them. For intersecting polyhedra, the origin is contained in theMinkowski sum of A and −B, and the penetration depth is equal to the minimum distance from the originto the surface of the Minkowski sum. The computation of penetration depth can be Ω(m 3 n 3 ) for generalpolyhedra [Dobkin et al. 1993].Many researchers have restricted the computation of penetration depth to convex polyhedra. In computationalgeometry, Dobkin et al. [Dobkin et al. 1993] presented an algorithm for computing directionalpenetration depth, while Agarwal et al. [Agarwal et al. 2000] introduced a randomized algorithm for computingthe penetration depth between convex polyhedra. Cameron [Cameron 1997] extended the GJKalgorithm [Gilbert et al. 1988] to compute bounds of the penetration depth, and van den Bergen [van denBergen 2001] furthered his work. Kim et al. [Kim et al. 2002a] presented an algorithm that computes alocally optimal solution of the penetration depth by walking on the surface of the Minkowski sum.The fastest algorithms for computation of penetration depth between arbitrary polyhedra take advantageof discretization. Fisher and Lin [Fisher and Lin 2001] estimate penetration depth using distance fieldscomputed with fast marching level-sets. Hoff et al. [Hoff et al. 2001] presented an image-based algorithmimplemented on graphics hardware. On the other hand, Kim et al. [Kim et al. 2002b] presented analgorithm that decomposes the polyhedra into convex patches, computes the Minkowski sums of pairwisepatches, and then uses an image-based technique in order to find the minimum distance from the origin tothe surface of the Minkowski sums.5.3 Hierarchical Collision DetectionThe algorithms for collision detection between convex polyhedra are not directly applicable to non-convexpolyhedra or models described as polygon soups. Brute force checking of all triangle pairs, however,is usually unnecessary. Collision detection between general models achieves large speed-ups by usinghierarchical culling or spatial partitioning techniques that restrict the primitive-level tests. Over the lastdecade, bounding volume hierarchies (BVH) have proved successful in the acceleration of collision detectionfor dynamic scenes of rigid bodies. For an extensive description and analysis of the use of BVHs for collisiondetection, please refer to Gottschalk’s PhD dissertation [Gottschalk 2000].A14

5 COLLISION DETECTION 13Assuming that an object is described by a set of triangles T, a BVH is a tree of BVs, where each BVC i bounds a cluster of triangles T i ∈ T. The clusters bounded by the children of C i constitute a partitionof T i . The effectiveness of a BVH is conditioned by ensuring that the branching factor of the tree is O(1)and that the size of the leaf clusters is also O(1). Often, the leaf BVs bound only one triangle. A BVHmay be created in a top-down manner, by successive partitioning of clusters, or in a bottom-up manner,by using merging operations.In order to perform interference detection using BVHs, two objects are queried by recursively traversingtheir BVHs in tandem. Each recursive step tests whether a pair of BVs a and b, one from each hierarchy,overlap. If a and b do not overlap, the recursion branch is terminated. Otherwise, if they overlap, thealgorithm is applied recursively to their children. If a and b are both leaf nodes, the triangles within themare tested directly. This process can be generalized to other types of proximity queries as well.One determining factor in the design of a BVH is the selection of the type of BV. Often there is atrade-off among the tightness of the BV (and therefore the culling efficiency), the cost of the collisiontest between two BVs, and the dynamic update of the BV (relevant for deformable models). Some ofthe common BVs, sorted approximately according to increasing query time, are: spheres [Quinlan 1994;Hubbard 1994], axis-aligned bounding boxes (AABB) [Beckmann et al. 1990], oriented bounding boxes(OBB) [Gottschalk et al. 1996], k-discrete-orientation polytopes (k-DOP) [Klosowski et al. 1998], convexhulls [Ehmann and Lin 2001], and swept sphere volumes (SSV) [Larsen et al. 2000]. BVHs of rigid bodiescan be computed as a preprocessing step, but deformable models require a bottom-up update of the BVsafter each deformation. Recently, James and Pai [James and Pai 2004] have presented the BD-tree, avariant of the sphere-tree data structure [Quinlan 1994] that can be updated in a fast top-down manner ifthe deformations are described by a small number of parameters.5.4 Multiresolution Collision DetectionMultiresolution analysis of a function decomposes the function into a basic low-resolution representationand a set of detail terms at increasing resolutions. Wavelets provide a mathematical framework for definingmultiresolution analysis [Stollnitz et al. 1996].Multiresolution representations of triangles meshes have drawn important attention in computer graphics.They have been defined in two major ways: following the mathematical framework of wavelets andsubdivision surfaces [Lounsbery et al. 1997; Eck et al. 1995] or following level-of-detail (LOD) simplificationtechniques (please refer to [Luebke et al. 2002] for a survey on the topic). LOD techniques present theadvantage of being applicable to arbitrary meshes, but they lack a well-defined metric of resolution. Theyconstruct the multiresolution representations starting from full-resolution meshes and applying sequencesof local simplification operations. LOD techniques can be divided into those that produce a discrete setof representations (static LODs), and those that produce continuously adaptive representations (dynamicLODs). Multiresolution or LOD techniques have been used in applications such as view-dependent rendering[Hoppe 1997; Luebke and Erikson 1997], interactive editing of meshes [Zorin et al. 1997], or real-timedeformations [Debunne et al. 2001]. The idea behind multiresolution techniques is to select the resolutionor LOD of the representation in an adaptive manner based on perceptual parameters, availability ofcomputational resources, and so forth.Multiresolution collision detection refers to the execution of approximate collision detection queriesusing adaptive object representations. Hubbard [Hubbard 1994] introduced the idea of using spheretrees[Quinlan 1994] for multiresolution collision detection, refining the BVHs in a breadth-first manneruntil the time allocated for collision detection expires. In a sphere-tree each level of the BVH can beregarded as an implicit approximation of the given mesh, by defining the surface as a union of spheres.Unlike LOD techniques, in which simplification operations minimize surface deviation, sphere-trees addextraneous “bumpiness” to the surface, and this characteristic can hurt collision response.O’Sullivan and Dingliana [O’Sullivan and Dingliana 2001] have incorporated perceptual parameters intoA15

6 RIGID BODY SIMULATION 14the refinement of sphere-trees. They insert pairs of spheres that test positive for collision in a priorityqueue sorted according to perceptual metrics (e.g., local relative velocity, distance to the viewer, etc.). Inthis way the adaptive refinement focuses on areas of the objects where errors are most noticeable.The use of multiresolution representations for haptic rendering has also been investigated by severalresearchers. Pai and Reissel [Pai and Reissel 1997] investigated the use of multiresolution image curvesfor 2D haptic interaction. El-Sana and Varshney [El-Sana and Varshney 2000] applied LOD techniques to3-DoF haptic rendering. They created a multiresolution representation of the haptically rendered object asa preprocessing step and, at runtime, they represented the object at high resolution near the probe pointand at low resolution further away. Their approach does not extend naturally to the interaction betweentwo objects, since multiple disjoint contacts can occur simultaneously at widely varying locations withoutmuch spatial coherence. Otaduy and Lin [Otaduy and Lin 2003b; Otaduy and Lin 2003a] introducedcontact levels of detail, dual hierarchical representations for multiresolution collision detection, and theyapplied them to 6-DoF haptic rendering, producing a sensation-preserving simplified rendering.5.5 Other Techniques for Collision DetectionWe briefly cover two additional topics with potential applicability in haptic rendering: the use of graphicsprocessors for collision detection, and continuous collision detection.5.5.1 Use of Graphics Processors for Collision DetectionThe processing capability of GPUs is growing at a rate higher than Moore’s law [Govindaraju et al.2003], and this circumstance has generated an increasing use of GPUs for general-purpose computation,including collision detection. Rasterization hardware enables high performance of image-based collisiondetection algorithms. Hoff et al. [Hoff et al. 2001] presented an algorithm for estimating penetration depthbetween deformable polygons using distance fields computed on graphics hardware. Others have formulatedcollision detection queries as visibility problems. Lombardo et al. [Lombardo et al. 1999] intersected acomplex object against a simpler one using the view frustum and clipping planes, and they detectedintersecting triangles by exploiting OpenGL capabilities. More recently, Govindaraju et al. [Govindarajuet al. 2003] have designed an algorithm that performs series of visibility queries and achieves fast cullingof non-intersecting primitives in N-body problems with nonrigid motion.5.5.2 Continuous Collision DetectionContinuous collision detection refers to a temporal formulation of the collision detection problem. Thecollision query attempts to find intersecting triangles and the time of intersection. Redon et al. [Redonet al. 2002] proposed an algorithm that assumes an arbitrary interframe rigid motion and incorporates thetemporal dimension in OBB-trees using interval arithmetic. Continuous collision detection offers potentialapplicability to haptic rendering because it may enable constraint-based simulations without expensivebacktracking operations used for computing the time of first collision.6 Rigid Body SimulationComputation of the motion of a rigid body consists of solving a set of ordinary differential equations(ODEs). The most common way to describe the motion of a rigid body is by means of the Newton-Eulerequations, which define the time derivatives of the linear momentum, P, and angular momentum, L, as afunction of external force F and torque T:A16

6 RIGID BODY SIMULATION 15F(t) =Ṗ(t) = m ẍ(t),T(t) = ˙L(t) = ω(t) × (Mω(t)) + M ˙ω(t). (2)As shown in the equations, momentum derivatives can be expressed in terms of the linear acceleration of thecenter of mass ẍ, the angular velocity ω, the mass of the body m, and the mass matrix M. The complexityof rigid body simulation lies in the computation of force and torque resulting from contacts between bodies.Research in the field of rigid body simulation has revolved around different methods for computing contactforces and the resulting accelerations and velocities, ranging from approximate methods that consider eachcontact independently (such as penalty-based methods) to analytic methods that account concurrently forall non-penetration constraints. Important efforts have been devoted to capturing friction forces as well.In this section we briefly describe the main methods for solving the motion of colliding rigid bodies,focusing on their applicability to haptic rendering. For further information, please refer to Baraff’s orMirtich’s dissertations [Baraff 1992; Mirtich 1996], SIGGRAPH course notes on the topic [Baraff andWitkin 2001], or recent work by Stewart and Trinkle [Stewart and Trinkle 2000]. In the last few years,especially in the field of computer graphics, attention has been drawn towards the problem of simulatingthe interaction of many rigid bodies [Mirtich 2000; Milenkovic and Schmidl 2001; Guendelman et al. 2003].For haptic rendering, however, one is mostly concerned with the dynamics of the object grasped by theuser; therefore the interaction of many rigid objects is not discussed here.6.1 Penalty-Based MethodsWhen two objects touch or collide, collision response must be applied to prevent object interpenetration.One method for implementing collision response is the insertion of stiff springs at the points ofcontact [Moore and Wilhelms 1988]. This method is inspired by the fact that, when objects collide, smalldeformations take place at the region of contact, and these deformations can be modeled with springs,even if the objects are geometrically rigid.Given two intersecting objects A and B, penalty-based collision response requires the definition of acontact point p, a contact normal n and a penetration depth δ. The penalty-based spring force and torqueapplied to object A are defined as follows:F A = −f(δ)n,T A = (p − c A ) × F A , (3)where c A is the center of mass of A. Opposite force and torque are applied to object B. The function fcould be a linear function defined by a constant stiffness k or a more complicated non-linear function. Itcould also contain a viscous term, dependent on the derivative of the penetration depth.The basic formulation of penalty methods can be modified slightly in order to introduce repulsive forcesbetween objects, by inserting contact springs when the objects come closer than a distance tolerance d. Inthis way, object interpenetration occurs less frequently. The addition of a tolerance has two major advantages:the possibility of using penalty methods in applications that do not allow object interpenetration,and a reduction of the cost of collision detection. As noted in Sec. 5, computation of penetration depth isnotably more costly than computation of separation distance.Penalty-based methods offer several attractive properties: the force model is local to each contact andcomputationally simple, object interpenetration is inherently allowed, and contact determination needsto be performed only once per simulation frame. This last property makes penalty-based methods bestsuited for interactive applications with fixed time steps, such as haptic rendering [McNeely et al. 1999; Kimet al. 2003; Johnson and Willemsen 2003; Otaduy and Lin 2005] and games [Wu 2000; Larsen 2001]. ButA17

6 RIGID BODY SIMULATION 16penalty-based methods also have some disadvantages. There is no direct control over physical parameters,such as the coefficient of restitution. Non-penetration constraints are enforced by means of very highcontact stiffness, and this circumstance leads to instability problems if numerical integration is executedusing fast, explicit methods. The solution of penalty-based simulation using implicit integration, however,enhances stability in the presence of high contact stiffness [Wu 2000; Larsen 2001; Otaduy and Lin 2005].Friction effects can be incorporated into penalty-based methods by means of localized force models thatconsider each contact point independently. Most local friction methods propose different force models forstatic or dynamic situations [Karnopp 1985; Hayward and Armstrong 2000]. Static friction is modeledby fixing adhesion points on the surfaces of the colliding objects and setting tangential springs betweenthe contact points and the adhesion points. If the elastic friction force becomes larger than a thresholddetermined by the normal force and the friction coefficient, the system switches to dynamic mode. In thedynamic mode, the adhesion point follows the contact point. The system returns to static mode if thevelocity falls under a certain threshold.So far, we have analyzed contact determination and collision response as two separate problems, but theoutput of the contact determination step has a strong influence on the smoothness of collision response and,as a result, on the stability of numerical integration. As pointed out by Larsen [Larsen 2001], when a newcontact point is added, the associated spring must be unstretched. In other words, the penetration depthvalue must be zero initially and must grow smoothly. The existence of geometry-driven discontinuities isan inherent problem of penalty-based simulations with fixed time steps. Some authors [Hasegawa and Sato2004] have proposed sampling the intersection volume to avoid geometric discontinuities in the applicationof penalty-based methods to rigid body simulation and haptic rendering, but this approach is applicableonly to very simple objects.6.2 Constraint-Based SimulationConstraint-based methods for the simulation of rigid body dynamics handle all concurrent contacts in asingle computational problem and attempt to find contact forces that produce physically and geometricallyvalid motions. Specifically, they integrate the Newton-Euler equations of motion (see Eq. 2), subject togeometric constraints that prevent object interpenetration. The numerical integration of Newton-Eulerequations must be interrupted before objects interpenetrate. At a collision event, object velocities and accelerationsmust be altered, so that non-penetration constraints are not violated and numerical integrationcan be restarted. One must first compute contact impulses that produce constraint-valid velocities. Then,one must compute contact forces that produce valid accelerations.The relative normal accelerations a at the points of contact can be expressed as linear combinations ofthe contact forces F (with constant matrix A and vector b). Moreover, one can impose non-penetrationconstraints on the accelerations and non-attraction constraints on the forces:a = AF + b,a ≥ 0, F ≥ 0. (4)Baraff [Baraff 1989] pioneered the application of constraint-based approaches to rigid body simulationin computer graphics. He posed constrained rigid body dynamics simulation as a quadratic programmingproblem on the contact forces, and he proposed a fast, heuristic-based solution for the frictionless case. Hedefined a quadratic cost function based on the fact that contact forces occur only at contact points thatare not moving apart:min ( F T a ) = min ( F T AF + F T b ) . (5)The quadratic cost function suggested by Baraff indicates that either the normal acceleration or thecontact force should be 0 at a resting contact. As indicated by Cottle et al. [Cottle et al. 1992], thisA18

7 HAPTIC TEXTURE RENDERING 17condition can be formulated as a linear complementarity problem (LCP). Baraff [Baraff 1991; Baraff 1992]added dynamic friction to the formulation of the problem and suggested approaches for static friction, aswell as a solution following an algorithm by Lemke [Lemke 1965] with expected polynomial cost in thenumber of constraints. Earlier, Lötstedt had studied the problem of rigid body dynamics with frictionin the formulation of the LCP [Lötstedt 1984]. Later, Baraff himself [Baraff 1994] adapted an algorithmby Cottle and Dantzig [Cottle and Dantzig 1968] for solving frictionless LCPs to the friction case, andachieved linear-time performance in practice.Stewart and Trinkle [Stewart and Trinkle 1996] presented an implicit LCP formulation of constraintbasedproblems. Unlike previous algorithms, which enforced the constraints only at the beginning of eachtime step, their algorithm solves for contact impulses that also enforce the constraints at the end of thetime step. This formulation eliminates the need to locate collision events, but it increases the number ofconstraints to be handled, and it is unclear how it behaves with complex objects.Stewart and Trinkle [Stewart and Trinkle 1996] mention the existence of geometry-driven discontinuities,similar to the ones appearing with penalty methods, in their implicit formulation of the LCP. Afternumerical integration of object position and velocities, new non-penetration constraints are computed. Ifnumerical integration is not interrupted at collision events, the newly computed non-penetration constraintsmay not hold. Constraint violation may produce unrealistically high contact impulses and object velocitiesin the next time step. This phenomenon is equivalent to the effect of prestretched penalty-based springsdescribed by Larsen [Larsen 2001]. Stewart and Trinkle suggest solving a non-linear complementarityproblem, with additional cost involved.If numerical integration is interrupted at collision events, the effects of geometry-driven discontinuitiescan be alleviated by capturing all the contact points that bound the contact region. Baraff [Baraff 1989]considers polygonal contact regions between polyhedral models and defines contact constraints at thevertices that bound the polygonal regions. Similarly, Mirtich [Mirtich 1998a] describes polygonal contactareas as combinations of edge-edge and vertex-face contacts.6.3 Impulse-Based DynamicsMirtich [Mirtich and Canny 1995; Mirtich 1996] presented a method for handling collisions in rigid bodydynamics simulation based solely on the application of impulses to the objects. In situations of resting,sliding, or rolling contact, constraint forces are replaced by trains of impulses. Mirtich defined a collisionmatrix that relates contact impulse to the change in relative velocity at the contact. His algorithmdecomposes the collision event into two separate processes: compression and restitution. Each processis parameterized separately, and numerical integration is performed in order to compute the velocitiesafter the collision. The parameterization of the collision event enables the addition of a friction model toinstantaneous collisions.The time-stepping engine of impulse-based dynamics is analogous to the one in constraint-based dynamics:numerical integration must be interrupted before interpenetration occurs, and valid velocities must becomputed. One of the problems of impulse-based dynamics emerges during inelastic collisions from the factthat accelerations are not recomputed. The energy loss induced by a train of inelastic collisions reducesthe time between collisions and increases the cost of simulation per frame. In order to handle this problem,Mirtich suggested the addition of unrealistic, but visually imperceptible, energy to the system when the microcollisionsbecome too frequent. As has been pointed out by Mirtich, impulse-based approaches are bestsuited for simulations that are collision-intensive, with multiple, different impacts occurring frequently.7 Haptic Texture RenderingAlthough haptic rendering of textures was one of the first tackled problems [Minsky et al. 1990], it has beenmostly limited to the interaction between a probe point and a textured surface. We begin this section withA19

7 HAPTIC TEXTURE RENDERING 18a description of Minsky’s pioneering algorithm for rendering textures on the plane [Minsky 1995]. Thenwe discuss rendering of textures on 3D surfaces, covering basic 3-DoF haptic rendering, height-field-basedmethods, and probabilistic methods.7.1 Rendering Textures on the PlaneMinsky [Minsky 1995] developed the Sandpaper system for 2-DoF haptic rendering of textures on a planarsurface. Her system was built around a force model for computing 2D forces from texture height fieldinformation. Following energy-based arguments, her force model synthesizes a force F in 2D based on thegradient of the texture height field h at the location of the probe:F = −k∇h. (6)Minsky also analyzed qualitatively and quantitatively roughness perception and the believability of theproposed force model. One of the main conclusions of her work is to establish her initial hypothesis, thattexture information can be conveyed by displaying forces tangential to the contact surface. This hypothesiswas later exploited for rendering textured 3D surfaces [Ho et al. 1999].7.2 3-Degree-of-Freedom Haptic RenderingAs described in Sec. 1.4, 3-DoF haptic rendering methods compute feedback force as a function of theseparation between the probe point controlled with the haptic device and a contact point constrainedto the surface of the haptically rendered object. Early 3-DoF haptic rendering methods set the contactpoint as the point on the surface of the object closest to the probe point. As has been addressed byZilles and Salisbury [Zilles and Salisbury 1995], these methods lead to force discontinuities and possible“pop-through” problems, in which the contact point jumps between opposing sides of the object. Instead,Zilles and Salisbury proposed the god-object method, which defines the computation of the contact point asa constrained optimization problem. The contact point is located at a minimum distance from the probepoint, but its interframe trajectory is constrained by the surface. Zilles and Salisbury solve the position ofthe contact point using Lagrange multipliers, once they define the set of active constraints.Ruspini et al. [Ruspini et al. 1997] followed a similar approach. They modeled the contact point as asphere of small radius and solved the optimization problem in the configuration space. Ruspini and hiscolleagues also added other effects, such as force shading for rounding of corners (by modifying the normalsof constraint planes), or friction (by adding dynamic behavior to the contact point).7.3 Methods Based on Height FieldsHigh-resolution surface geometry can be represented by a parameterized coarse mesh along with textureimages storing detailed height field or displacement field information, similarly to the common approach oftexture mapping in computer graphics [Catmull 1974]. Constraint-based 3-DoF haptic rendering methodsdetermine a unique contact point on the surface of the rendered object. Usually, the mesh representationused for determining the contact point is rather coarse and does not capture high-frequency texture.Nevertheless, the parametric coordinates of the contact point can be used for accessing surface textureinformation from texture images.Ho et al. [Ho et al. 1999] introduced a technique similar to bump mapping [Blinn 1978] that alters thesurface normal based on the gradient of the texture height field. A combination of the original and refinednormals is used for computing the direction of the feedback force.Techniques for haptic texture rendering based on a single contact point can capture geometric propertiesof only one object and are not suitable for simulating full interaction between two surfaces. The geometricinteraction between two surfaces is not limited to, and cannot be described by, a pair of contact points.A20

8 6-DEGREE-OF-FREEDOM HAPTIC RENDERING 19Moreover, the local kinematics of the contact between two surfaces include rotational degrees of freedom,which are not captured by point-based methods.Ho et al. [Ho et al. 1999] indicate that a high height field gradient can induce system instability. Alonga similar direction, Choi and Tan [Choi and Tan 2003b; Choi and Tan 2003a] have studied the influence ofcollision detection and penetration depth computation on 3-DoF haptic texture rendering. Discontinuitiesin the output of collision detection are perceived by the user, a phenomenon that they describe as aliveness.This phenomenon is a possible problem in 6-DoF haptic rendering too.7.4 Probabilistic MethodsSome researchers have exploited statistical properties of surfaces for computing texture-induced forces thatare added to the classic 3-DoF contact forces. Siira and Pai [Siira and Pai 1996] synthesized texture forcesaccording to a Gaussian distribution for generating a sensation of roughness. In order to improve stability,they did not apply texture forces during static contact. Later, Pai et al. [Pai et al. 2001] presented atechnique for rendering roughness effects by dynamically modifying the coefficient of friction of a surface.The roughness-related portion of the friction coefficient was computed according to an autoregressiveprocess driven by noise.Probabilistic methods have proved to be successful for rendering high-frequency roughness effects inpoint-surface contact. It is also possible, although this approach has yet to be explored, that they couldbe combined with geometric techniques for synthesizing high-frequency effects in 6-DoF haptic rendering.8 6-Degree-of-Freedom Haptic RenderingThe problem of 6-DoF haptic rendering has been studied by several researchers. As introduced in Sec. 2.1,the existing methods for haptic rendering can be classified into two large groups based on their overallpipelines: direct rendering methods and virtual coupling methods. Each group of methods presents someadvantages and disadvantages. Direct rendering methods are purely geometric, and there is no need tosimulate the rigid body dynamics of the grasped object. However, penetration values may be quite largeand visually perceptible, and system instability can arise if the force update rate drops below the rangeof stable values. Virtual coupling methods enable reduced interpenetration, higher stability, and highercontrol of the displayed stiffness. However, virtual coupling [Colgate et al. 1995] may introduce noticeablefiltering, both tactile and visual, and it requires the simulation of rigid body dynamics.The different 6-DoF haptic rendering methods propose a large variety of options for solving the specificproblems of collision detection, collision response, and simulation of rigid body dynamics. In the presenceof infinite computational resources, an ideal approach to the problem of 6-DoF haptic rendering would be tocompute the position of the grasped object using constraint-based rigid body dynamics simulation [Baraff1992] and to implement force feedback through virtual coupling. This approach has indeed been followedby some, but it imposes serious limitations on the complexity of the objects and contact configurationsthat can be handled interactively. We now discuss briefly the different existing methods for 6-DoF hapticrendering, focusing on those that have been applied to moderately complex objects and scenarios.8.1 Direct Haptic Rendering ApproachesGregory et al. [Gregory et al. 2000b] presented a 6-DoF haptic rendering system that combined collisiondetection based on convex decomposition of polygonal models [Ehmann and Lin 2001], predictive estimationof penetration depth, and force and torque interpolation. They were able to handle interactively dynamicscenes with several convex objects, as well as pairs of non-convex objects with a few hundred trianglesand rather restricted motion. Kim et al. [Kim et al. 2003] exploited convex decomposition for collisiondetection and incorporated fast, incremental, localized computation of per-contact penetration depth [KimA21

8 6-DEGREE-OF-FREEDOM HAPTIC RENDERING 20et al. 2002a]. In order to improve stability and eliminate the influence of triangulation on the descriptionof the contact manifold, they introduced a contact clustering technique. Their system was able to handlepairs of models with nearly one hundred convex pieces each interactively.Earlier, Nelson et al. [Nelson et al. 1999] introduced a technique for haptic interaction between pairs ofparametric surfaces. Their technique tracks contact points that realize locally maximum penetration depthduring surface interpenetration. Tracking contact points, instead of recomputing them for every frame,ensures smooth penetration values, which are used for penalty-based force feedback. The contact pointsare solved in parametric space, and they are defined as those pairs of points for which their differencevector is collinear with surface normals.Johnson and Willemsen [Johnson and Willemsen 2003] suggested a technique for polygonal models thatdefines contact points as those that satisfy a local minimum-distance criterion, according to Nelson’sdefinition [Nelson et al. 1999]. Johnson and Willemsen exploit this definition in a fast collision cullingalgorithm, using spatialized normal cone hierarchies [Johnson and Cohen 2001]. The performance oftheir technique depends on the convexity and triangulation of the models, which affect the number ofcontact points. Recently, Johnson and Willemsen [Johnson and Willemsen 2004] have incorporated anapproximate but fast, incremental contact-point-tracking algorithm that is combined with slower exactcollision updates from their previous technique [Johnson and Willemsen 2003]. This algorithm handlesmodels with thousands of triangles at interactive rates, but the forces may suffer discontinuities if theexact update is too slow.8.2 Virtual Coupling with Object VoxelizationIn 1999, McNeely et al. [McNeely et al. 1999] presented a system for 6-DoF haptic rendering that employsa discrete collision detection approach and virtual coupling. The system is intended for assembly andmaintenance planning applications and assumes that only one of the objects in the scene is dynamic.The surfaces of the scene objects are voxelized, and the grasped object is point-sampled. The collisiondetection module checks for inclusion of the sample points in the scene voxels, and then a local forcemodel is applied. Hierarchical culling of sample points is possible, but ultimately the computational costdepends on the number of contact points. This system has been integrated in a commercial product, VPS,distributed by Boeing.McNeely and his colleagues introduced additional features in order to alleviate some of the limitations.Surface objects are voxelized only on the surface, therefore deep penetrations, which can occur if objectscollide at high velocities, cannot be handled. They propose pre-contact braking forces, similar to thebraking impulses suggested by Salcudean [Salcudean and Vlaar 1994], for reducing the contact velocity ofthe grasped object and thereby preventing deep penetrations. The existence of multiple contact pointsproduces high stiffness values that can destabilize the simulation of rigid body dynamics. They proposeaveraging the effects of the different contact points before contact forces are applied to the grasped object,for limiting the stiffness and thereby ensuring stable simulation. The locality of the force model inducesforce discontinuities when contact points traverse voxel boundaries. They point out that force discontinuitiesare somewhat filtered by the virtual coupling. Renz et al. [Renz et al. 2001] modified McNeely’s localforce model to ensure continuity of the surface across voxel boundaries, but incurring more expensive forcecomputation.Using the same voxelization and point-sampling approach for collision detection, Wan and McNeely [Wanand McNeely 2003] have proposed a novel solution for computing the position of the grasped object. Theearly approach by McNeely et al. [McNeely et al. 1999] computed object dynamics by explicit integrationof Newton-Euler equations. Instead, Wan and McNeely [Wan and McNeely 2003] presented a purelygeometric solution that eliminates the instability problems that can arise due to high contact stiffness.Their algorithm formulates linear approximations of the coupling and contact force and torque in thespace of translations and rotations of the grasped object. The state of the object is computed at everyA22

REFERENCES 21frame by solving for the position of quasi-static equilibrium. Deep penetrations are avoided by formulatingthe coupling force as a non-linear spring.8.3 Rigid Body Dynamics with Haptic FeedbackChang and Colgate [Chang and Colgate 1997] proposed a solution to 6-DoF haptic rendering by combiningvirtual coupling [Colgate et al. 1995] and rigid body simulation based on impulse dynamics [Mirtich 1996].They found that impulses alone were not efficient in resting contact situations, and in those cases theysuggested a combination of impulses and penalty forces. Recently, Constantinescu et al. [Constantinescuet al. 2004] have reached a similar conclusion. As has been addressed by Constantinescu, combiningimpulses and penalty forces requires a state machine in order to determine the state of the object, but itis not clear how to extend this solution to scenes with many contacts. Both Chang and Constantinescuhave tested their implementations only on simple benchmarks.One of the reasons for the simplicity of Chang and Constantinescu’s benchmarks is the cost of collisiondetection for the simulation of rigid body dynamics. As has been discussed in Sec. 6, impulse- [Mirtich 1996]or constraint-based [Baraff 1992] methods must interrupt the integration before object interpenetration,and this leads to many collision queries per frame. Some researchers have integrated haptic interactionwith constraint-based rigid body simulations [Berkelman 1999; Ruspini and Khatib 2000] in scenes withsimple geometry.As indicated in Sec. 6.1, non-penetration constraints can be relaxed using penalty-based methods. Mc-Neely et al. [McNeely et al. 1999] employed penalty methods for rigid body simulation but, as explainedearlier, they observed numerical instabilities due to high stiffness values, and large interpenetrations underhigh impact velocities. Those problems can be tackled with high-stiffness penalty contact forces alongwith implicit integration, an approach used in interactive rigid body simulations [Wu 2000; Larsen 2001].Implicit integration requires the evaluation of the Jacobian of the Newton-Euler equations and the solutionof a linear system of equations [Baraff and Witkin 1998]. As demonstrated by Otaduy and Lin [Otaduyand Lin 2005], implicit integration can be performed at force update rates under the assumption that onlythe grasped object is dynamic.8.4 Multiresolution TechniquesThe application of 6-DoF haptic rendering algorithms to complex models and complex contact scenariosbecomes a challenging issue, due to the inherent cost of collision detection that induces slow force updates.Otaduy and Lin [Otaduy and Lin 2003b] have presented a sensation-preserving simplification technique for6-DoF haptic rendering of complex polygonal models by selecting contact resolutions adaptively. Otaduyet al. [Otaduy et al. 2004] have also proposed a rendering algorithm for the interaction of textured surfaces.Their work is focused on the acceleration of collision detection and response using level-of-detailrepresentations and texture images.ReferencesAdachi, Y., Kumano, T., and Ogino, K. 1995. Intermediate representation for stiff virtual objects.Virtual Reality Annual International Symposium, 203–210.Adams, R. J., and Hannaford, B. 1998. A two-port framework for the design of unconditionally stablehaptic interfaces. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems.Agarwal, P., Guibas, L., Har-Peled, S., Rabinovitch, A., and Sharir, M. 2000. Penetrationdepth of two convex polytopes in 3d. Nordic J. Computing 7, 227–240.Andriot, C. 2002. Advances in virtual prototyping. Clefs CEA Vol. 47, Research and Simulation.A23

REFERENCES 22Astley, O. R., and Hayward, V. 1998. Multirate haptic simulation achieved by coupling finiteelement meshes through norton equivalents. Proc. of IEEE International Conference on Robotics andAutomation.Avila, R. S., and Sobierajski, L. M. 1996. A haptic interaction method for volume visualization. InIEEE Visualization ’96, IEEE. ISBN 0-89791-864-9.Baraff, D., and Witkin, A. 1998. Large steps in cloth simulation. Proc. of ACM SIGGRAPH, 43–54.Baraff, D., and Witkin, A. 2001. Physically-Based Modeling. ACM SIGGRAPH Course Notes.Baraff, D. 1989. Analytical methods for dynamic simulation of non-penetrating rigid bodies. In ComputerGraphics (SIGGRAPH ’89 Proceedings), J. Lane, Ed., vol. 23, 223–232.Baraff, D. 1991. Coping with friction for non-penetrating rigid body simulation. In Computer Graphics(SIGGRAPH ’91 Proceedings), T. W. Sederberg, Ed., vol. 25, 31–40.Baraff, D. 1992. Dynamic simulation of non-penetrating rigid body simulation. PhD thesis, CornellUniversity.Baraff, D. 1994. Fast contact force computation for nonpenetrating rigid bodies. In Proceedings ofSIGGRAPH ’94, A. Glassner, Ed., ACM SIGGRAPH, 23–34. ISBN 0-89791-667-0.Basdogan, C., Ho, C.-H., and Srinivasan, M. A. 1997. A ray-based haptic rendering technique fordisplaying shape and texture of 3d objects in virtual environments. DSC-Vol. 61, Proceedings of theASME Dynamic Systems and Control Division, pp. 77–84.Beckmann, N., Kriegel, H., Schneider, R., and Seeger, B. 1990. The r*-tree: An efficient androbust access method for points and rectangles. Proc. SIGMOD Conf. on Management of Data, 322–331.Bejczy, A., and Salisbury, J. K. 1980. Kinematic coupling between operator and remote manipulator.Advances in Computer Technology Vol. 1, 197–211.Berkelman, P. J. 1999. Tool-Based Haptic Interaction with Dynamic Physical Simulations Using LorentzMagnetic Levitation. PhD thesis, Carnegie Mellon University.Blinn, J. F. 1978. Simulation of wrinkled surfaces. In Computer Graphics (SIGGRAPH ’78 Proceedings),286–292.Brooks, Jr., F. P., Ouh-Young, M., Batter, J. J., and Kilpatrick, P. J. 1990. Project GROPE— Haptic displays for scientific visualization. In Computer Graphics (SIGGRAPH ’90 Proceedings),F. Baskett, Ed., vol. 24, 177–185.Burdea, G. 1996. Force and Touch Feedback for Virtual Reality. John Wiley and Sons.Cameron, S., and Culley, R. K. 1986. Determining the minimum translational distance between twoconvex polyhedra. Proceedings of International Conference on Robotics and Automation, 591–596.Cameron, S. 1997. Enhancing GJK: Computing minimum and penetration distance between convexpolyhedra. IEEE International Conference on Robotics and Automation, 3112–3117.Catmull, E. E. 1974. A Subdivision Algorithm for Computer Display of Curved Surfaces. PhD thesis,Dept. of CS, U. of Utah.A24

REFERENCES 23Çavuşoğlu, M. C., and Tendick, F. 2000. Multirate simulation for high fidelity haptic interactionwith deformable objects in virtual environments. Proc. of IEEE International Conference on Roboticsand Automation, 2458–2465.Çavuşoğlu, M. C., Tendick, F., and Sastry, S. S. 2002. Haptic interfaces to real and virtualsurgical environments. In Touch in Virtual Environments, M. L. McLaughlin, J. P. Hespanha, and G. S.Sukhatme, Eds. Prentice Hall PTR, Upper Saddle River, NJ, ch. 13, 217–237.Chang, B., and Colgate, J. E. 1997. Real-time impulse-based simulation of rigid body systems forhaptic display. Proc. of ASME Dynamic Systems and Control Division.Chen, E. 1999. Six degree-of-freedom haptic system for desktop virtual prototyping applications. InProceedings of the First International Workshop on Virtual Reality and Prototyping, 97–106.Choi, S., and Tan, H. Z. 2003. Aliveness: Perceived instability from a passive haptic texture renderingsystem. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems.Choi, S., and Tan, H. Z. 2003. An experimental study of perceived instability during haptic texturerendering: Effects of collision detection algorithm. Proc. of Haptics Symposium, 197–204.Colgate, J. E., and Brown, J. M. 1994. Factors affecting the z-width of a haptic display. IEEEInternational Conference on Robotics and Automation, 3205–3210.Colgate, J. E., and Schenkel, G. G. 1994. Passivity of a class of sampled-data systems: Applicationto haptic interfaces. Proc. of American Control Conference.Colgate, J. E., Grafing, P. E., Stanley, M. C., and Schenkel, G. 1993. Implementation of stiffvirtual walls in force-reflecting interfaces. Virtual Reality Annual International Symposium, 202–207.Colgate, J. E., Stanley, M. C., and Brown, J. M. 1995. Issues in the haptic display of tool use.Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 140–145.Connor, C. E., and Johnson, K. O. 1992. Neural coding of tactile texture: Comparison of spatial andtemporal mechanisms for roughness perception. Journal of Neuroscience 12, pp. 3414–3426.Constantinescu, D., Salcudean, S. E., and Croft, E. A. 2004. Impulsive forces for haptic renderingof rigid contact. Proc. of International Symposium on Robotics, 1–6.Cottle, R. W., and Dantzig, G. B. 1968. Complementarity pivot theory of mathematical programming.Linear Algebra and its Applications, 103–125.Cottle, R. W., pang, J. S., and Stone, R. E. 1992. The linear complementarity problem. Academic-Press, Inc..Dachille, F., Qin, H., , Kaufman, A., and El-Sana, J. 1999. Haptic sculpting of dynamic surfaces.Proc. of ACM Symposium on Interactive 3D Graphics, 103–110.Debunne, G., Desbrun, M., Cani, M. P., and Barr, A. H. 2001. Dynamic real-time deformationsusing space and time adaptive sampling. Proc. of ACM SIGGRAPH.Dobkin, D. P., and Kirkpatrick, D. G. 1990. Determining the separation of preprocessed polyhedra –a unified approach. In Proc. 17th Internat. Colloq. Automata Lang. Program., Springer-Verlag, vol. 443of Lecture Notes in Computer Science, 400–413.Dobkin, D., Hershberger, J., Kirkpatrick, D., and Suri, S. 1993. Computing the intersectiondepthof polyhedra. Algorithmica 9, 518–533.A25

REFERENCES 24Duriez, C., Andriot, C., and Kheddar, A. 2004. A multi-threaded approach for deformable/rigidcontacts with haptic feedback. Proc. of Haptics Symposium.Eck, M., DeRose, T., Duchamp, T., Hoppe, H., Lounsbery, M., and Stuetzle, W. 1995.Multiresolution analysis of arbitrary meshes. In SIGGRAPH 95 Conference Proceedings, Addison Wesley,R. Cook, Ed., Annual Conference Series, ACM SIGGRAPH, 173–182. held in Los Angeles, California,06-11 August 1995.Edmond, C., Heskamp, D., Sluis, D., Stredney, D., Wiet, G., Yagel, R., Weghorst, S.,Oppenheimer, P., Miller, J., Levin, M., and Rosenberg, L. 1997. Ent endoscopic surgicalsimulator. Proc. of Medicine Meets VR, 518–528.Ehmann, S., and Lin, M. C. 2000. Accelerated proximity queries between convex polyhedra usingmulti-level voronoi marching. Proc. of IEEE/RSJ International Conference on Intelligent Robots andSystems, 2101–2106.Ehmann, S., and Lin, M. C. 2001. Accurate and fast proximity queries between polyhedra using convexsurface decomposition. Computer Graphics Forum (Proc. of Eurographics’2001) 20, 3, 500–510.El-Sana, J., and Varshney, A. 2000. Continuously-adaptive haptic rendering. Virtual Environments2000, pp. 135–144.Ellis, R. E., Sarkar, N., and Jenkins, M. A. 1997. Numerical methods for the force reflection ofcontact. ASME Transactions on Dynamic Systems, Modeling and Control 119, 768–774.Ernst, M. O., and Banks, M. S. 2001. Does vision always dominate haptics?Environments Conference.Touch in VirtualFisher, S., and Lin, M. C. 2001. Fast penetration depth estimation for elastic bodies using deformeddistance fields. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems 2001.Fisher, B., Fels, S., MacLean, K., Munzner, T., and Rensink, R. 2004. Seeing, hearing andtouching: Putting it all together. In ACM SIGGRAPH course notes.Foskey, M., Otaduy, M. A., and Lin, M. C. 2002. ArtNova: Touch-enabled 3D model design. Proc.of IEEE Virtual Reality Conference.Gibson, S., Samosky, J., Mor, A., Fyock, C., Grimson, E., and Kanade, T. 1997. Simulatingarthroscopic knee surgery using volumetric object representations, real-time volume rendering and hapticfeedback. First Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine andMedical Robotics and Computer-Assisted Surgery (CVMed-MRCAS), 368–378.Gilbert, E. G., Johnson, D. W., and Keerthi, S. S. 1988. A fast procedure for computing thedistance between objects in three-dimensional space. IEEE J. Robotics and Automation vol RA-4,193–203.Goertz, R., and Thompson, R. 1954. Electronically controlled manipulator. Nucleonics, 46–47.Gottschalk, S., Lin, M., and Manocha, D. 1996. OBB-Tree: A hierarchical structure for rapidinterference detection. Proc. of ACM Siggraph’96, 171–180.Gottschalk, S. 2000. Collision Queries using Oriented Bounding Boxes. PhD thesis, University ofNorth Carolina. Department of Computer Science.A26

REFERENCES 25Govindaraju, N., Redon, S., Lin, M., and Manocha, D. 2003. CULLIDE: Interactive collisiondetection between complex models in large environments using graphics hardware. Proc. of ACM SIG-GRAPH/Eurographics Workshop on Graphics Hardware, 25–32.Gregory, A., Lin, M., Gottschalk, S., and Taylor, R. 1999. H-COLLIDE: A framework for fastand accurate collision detection for haptic interaction. In Proceedings of Virtual Reality Conference 1999,38–45.Gregory, A., Ehmann, S., and Lin, M. C. 2000. inTouch: Interactive multiresolution modeling and3d painting with a haptic interface. Proc. of IEEE VR Conference.Gregory, A., Mascarenhas, A., Ehmann, S., Lin, M. C., and Manocha, D. 2000. 6-DoF hapticdisplay of polygonal models. Proc. of IEEE Visualization Conference.Guendelman, E., Bridson, R., and Fedkiw, R. 2003. Nonconvex rigid bodies with stacking. ACMTrans. on Graphics (Proc. of ACM SIGGRAPH) 22, 871–878.Guibas, L., Hsu, D., and Zhang, L. 1999. H-Walk: Hierarchical distance computation for movingconvex bodies. Proc. of ACM Symposium on Computational Geometry.Hannaford, B., Ryu, J.-H., and Kim, Y. S. 2002. Stable control of haptics. In Touch in VirtualEnvironments, M. L. McLaughlin, J. P. Hespanha, and G. S. Sukhatme, Eds. Prentice Hall PTR, UpperSaddle River, NJ, ch. 3, 47–70.Hasegawa, S., and Sato, M. 2004. Real-time rigid body simulation for haptic interactions based oncontact volume of polygonal objects. In Proc. of Eurographics.Hayward, V., and Armstrong, B. 2000. A new computational model of friction applied to hapticrendering. Experimental Robotics VI.Hayward, V., Gregorio, P., Astley, O., Greenish, S., and Doyon, M. 1998. Freedom-7: A highfidelity seven axis haptic device with applications to surgical training. Experimental Robotics, 445–456.Lecture Notes in Control and Information Sciences 232.Heller, M. A., Calcaterra, J. A., Green, S. L., and Brown, L. 1999. Intersensory conflictbetween vision and touch: The response modality dominates when precise, attention-riveting judgementsare required. Perception and Psychophysics 61, pp. 1384–1398.Hill, J. W., and Salisbury, J. K. 1977. Two measures of performance in a peg-in-hole manipulationtask with force feedback. Thirteenth Annual Conference on Manual Control, MIT.Ho, C.-H., Basdogan, C., and Srinivasan, M. A. 1999. Efficient point-based rendering techniquesfor haptic display of virtual objects. Presence 8, 5, pp. 477–491.Hoff, K., Zaferakis, A., Lin, M., and Manocha, D. 2001. Fast and simple 2d geometric proximityqueries using graphics hardware. Proc. of ACM Symposium on Interactive 3D Graphics, 145–148.Hogan, N. 1985. Impedance control: An approach to manipulation, part i - theory, part ii - implementation,part iii - applications. Journal of Dynamic Systems, Measurement and Control 107, 1–24.Hogan, N. 1986. Multivariable mechanics of the neuromuscular system. IEEE Annual Conference of theEngineering in Medicine and Biology Society, 594–598.Hollins, M., and Risner, S. 2000. Evidence for the duplex theory of tactile texture perception.Perception & Psychophysics 62, 695–705.A27

REFERENCES 26Hoppe, H. 1997. View dependent refinement of progressive meshes. In ACM SIGGRAPH ConferenceProceedings, 189–198.Hubbard, P. 1994. Collision Detection for Interactive Graphics Applications. PhD thesis, Brown University.Insko, B., Meehan, M., Whitton, M., and Brooks, F. 2001. Passive haptics significantly enhancesvirtual environments. Tech. Rep. 01-010, Department of Computer Science, UNC Chapel Hill.Insko, B. 2001. Passive Haptics Significantly Enhance Virtual Environments. PhD thesis, University ofNorth Carolina. Department of Computer Science.James, D. L., and Pai, D. K. 2004. Bd-tree: Output-sensitive collision detection for reduced deformablemodels. ACM Trans. on Graphics (Proc. of ACM SIGGRAPH).Johnson, D. E., and Cohen, E. 2001. Spatialized normal cone hierarchies. Proc. of ACM Symposiumon Interactive 3D Graphics, pp. 129–134.Johnson, D. E., and Willemsen, P. 2003. Six degree of freedom haptic rendering of complex polygonalmodels. In Proc. of Haptics Symposium.Johnson, D. E., and Willemsen, P. 2004. Accelerated haptic rendering of polygonal models throughlocal descent. Proc. of Haptics Symposium.Johnson, D., Thompson II, T. V., Kaplan, M., Nelson, D., and Cohen, E. 1999. Painting textureswith a haptic interface. Proceedings of IEEE Virtual Reality Conference.Karnopp, D. 1985. Computer simulation of stick slip friction in mechanical dynamic systems. Trans.ASME, Journal of Dynamic Systems, Measurement, and Control.Katz, D. 1989. The World of Touch. Erlbaum, Hillsdale, NJ. L. Krueger, Trans. (Original work published1925).Kilpatrick, P. J. 1976. The use of a kinesthetic supplement in an interactive graphics system. Ph.d.thesis, The University of North Carolina at Chapel Hill.Kim, W., and Bejczy, A. 1991. Graphical displays for operator aid in telemanipulation. IEEE InternationalConference on Systems, Man and Cybernetics.Kim, Y. J., Lin, M. C., and Manocha, D. 2002. DEEP: an incremental algorithm for penetrationdepth computation between convex polytopes. Proc. of IEEE Conference on Robotics and Automation,921–926.Kim, Y. J., Lin, M. C., and Manocha, D. 2002. Fast penetration depth computation using rasterizationhardware and hierarchical refinement. Proc. of Workshop on Algorithmic Foundations of Robotics.Kim, Y. J., Otaduy, M. A., Lin, M. C., and Manocha, D. 2003. Six-degree-of-freedom hapticrendering using incremental and localized computations. Presence 12, 3, 277–295.Klatzky, R. L., and Lederman, S. J. 1995. Identifying objects from a haptic glance. Perception andPsychophysics 57, pp. 1111–1123.Klatzky, R. L., and Lederman, S. J. 1999. Tactile roughness perception with a rigid link interposedbetween skin and surface. Perception and Psychophysics 61, pp. 591–607.A28

REFERENCES 27Klatzky, R. L., and Lederman, S. J. 2002. Perceiving texture through a probe. In Touch in VirtualEnvironments, M. L. McLaughlin, J. P. Hespanha, and G. S. Sukhatme, Eds. Prentice Hall PTR, UpperSaddle River, NJ, ch. 10, 180–193.Klatzky, R. L., and Lederman, S. J. 2003. Touch. In Experimental Psychology, 147–176. Volume 4in I.B. Weiner (Editor-in-Chief). Handbook of Psychology.Klatzky, R. L., Lederman, S. J., Hamilton, C., Grindley, M., and Swendsen, R. H. 2003. Feelingtextures through a probe: Effects of probe and surface geometry and exploratory factors. Perceptionand Psychophysics 65(4), pp. 613–631.Klosowski, J., Held, M., Mitchell, J., Sowizral, H., and Zikan, K. 1998. Efficient collisiondetection using bounding volume hierarchies of k-dops. IEEE Trans. on Visualization and ComputerGraphics 4, 1, 21–37.Kuhnapfel, U. G., Kuhn, C., Hubner, M., Krumm, H.-G., Maass, H., and Neisius, B. 1997. Thekarlsruhe endoscopic surgery trainer as an example for virtual reality in medical education. MinimallyInvasive Therapy and Allied Technologies Vol. 6, 122–125.LaMotte, R. H., and Srinivasan, M. A. 1991. Surface microgeometry: Tactile perception and neuralencoding. In Information Processing in the Somatosensory System, O. Franzen and J. Westman, Eds.Macmillan Press, London, 49–58.Larsen, E., Gottschalk, S., Lin, M., and Manocha, D. 2000. Distance queries with rectangularswept sphere volumes. Proc. of IEEE Int. Conference on Robotics and Automation.Larsen, E. 2001. A robot soccer simulator: A case study for rigid body contact. Game DevelopersConference.Lederman, S. J., Klatzky, R. L., Hamilton, C., and Ramsay, G. I. 1999. Perceiving roughnessvia a rigid stylus: Psychophysical effects of exploration speed and mode of touch. Haptics-e.Lederman, S. J., Klatzky, R. L., Hamilton, C., and Grindley, M. 2000. Perceiving surfaceroughness through a probe: Effects of applied force and probe diameter. Proceedings of the ASMEDSCD-IMECE.Lederman, S. J. 1974. Tactile roughness of grooved surfaces: The touching process and the effects ofmacro- and microsurface structure. Perception and Psychophysics 16, pp. 385–395.Lemke, C. E. 1965. Bimatrix equilibrium points and mathematical programming. Management Science11, 681–689.Lin, M. C., and Canny, J. F. 1991. Efficient algorithms for incremental distance computation. In Proc.IEEE Internat. Conf. Robot. Autom., vol. 2, 1008–1014.Lin, M., and Gottschalk, S. 1998. Collision detection between geometric models: A survey. Proc. ofIMA Conference on Mathematics of Surfaces.Lin, M. C., and Manocha, D. 2004. Collision and proximity queries. In Handbook of Discrete andComputational Geometry, 2nd Ed., J. E. Goodman and J. O’Rourke, Eds. CRC Press LLC, Boca Raton,FL, ch. 35, 787–807.Lin, M. 1993. Efficient Collision Detection for Animation and Robotics. PhD thesis, Department ofElectrical Engineering and Computer Science, University of California, Berkeley.A29

REFERENCES 28Lombardo, J. C., Cani, M.-P., and Neyret, F. 1999. Real-time collision detection for virtual surgery.Proc. of Computer Animation.Lötstedt, P. 1984. Numerical simulation of time-dependent contact friction problems in rigid bodymechanics. SIAM Journal of Scientific Statistical Computing 5, 370–393.Lounsbery, M., DeRose, T. D., and Warren, J. 1997. Multiresolution analysis for surfaces ofarbitrary topological type. ACM Trans. Graph. 16, 1 (Jan.), 34–73.Luebke, D., and Erikson, C. 1997. View-dependent simplification of arbitrary polygon environments.In Proc. of ACM SIGGRAPH.Luebke, D., Reddy, M., Cohen, J., Varshney, A., Watson, B., and Huebner, R. 2002. Level ofDetail for 3D Graphics. Morgan-Kaufmann.Mark, W., Randolph, S., Finch, M., Van Verth, J., and Taylor II, R. M. 1996. Addingforce feedback to graphics systems: Issues and solutions. In SIGGRAPH 96 Conference Proceedings,H. Rushmeier, Ed., Annual Conference Series, 447–452.Massie, T. M., and Salisbury, J. K. 1994. The phantom haptic interface: A device for probingvirtual objects. Proc. of ASME Haptic Interfaces for Virtual Environment and Teleoperator Systems 1,295–301.McDonnell, K., Qin, H., and Wlodarczyk, R. 2001. Virtual clay: A real-time sculpting systemwith haptic interface. Proc. of ACM Symposium on Interactive 3D Graphics, 179–190.McLaughlin, M., Hespanha, J. P., and Sukhatme, G. S. 2002. Touch in Virtual Environments.Prentice Hall.McNeely, W., Puterbaugh, K., and Troy, J. 1999. Six degree-of-freedom haptic rendering usingvoxel sampling. Proc. of ACM SIGGRAPH, 401–408.Milenkovic, V. J., and Schmidl, H. 2001. Optimization-based animation. SIGGRAPH 01 ConferenceProceedings, 37–46.Miller, B. E., Colgate, J. E., and Freeman, R. A. 1990. Guaranteed stability of haptic systemswith nonlinear virtual environments. IEEE Transactions on Robotics and Automation 16, 6, 712–719.Minsky, M., Ouh-young, M., Steele, O., Brooks, Jr., F. P., and Behensky, M. 1990. Feelingand seeing: Issues in force display. In Computer Graphics (1990 Symposium on Interactive 3D Graphics),R. Riesenfeld and C. Sequin, Eds., vol. 24, 235–243.Minsky, M. 1995. Computational Haptics: The Sandpaper System for Synthesizing Texture for a Force-Feedback Display. PhD thesis, Ph.D. Dissertation, Program in Media Arts and Sciences, MIT. Thesiswork done at UNC-CH Computer Science.Mirtich, B., and Canny, J. 1995. Impulse-based simulation of rigid bodies. In Symposium on Interactive3D Graphics, ACM Press.Mirtich, B. V. 1996. Impulse-based Dynamic Simulation of Rigid Body Systems. PhD thesis, Universityof California, Berkeley.Mirtich, B. V. 1998. Rigid body contact: Collision detection to force computation. Tech. Rep. TR98-01,Mitsubishi Electric Research Laboratory.A30

REFERENCES 29Mirtich, B. 1998. V-Clip: Fast and robust polyhedral collision detection. ACM Transactions on Graphics17, 3 (July), 177–208.Mirtich, B. 2000. Timewarp rigid body simulation. SIGGRAPH 00 Conference Proceedings, 193–200.Moore, M., and Wilhelms, J. 1988. Collision detection and response for computer animation. InComputer Graphics (SIGGRAPH ’88 Proceedings), J. Dill, Ed., vol. 22, 289–298.Nelson, D. D., Johnson, D. E., and Cohen, E. 1999. Haptic rendering of surface-to-surface sculptedmodel interaction. Proc. of ASME Dynamic Systems and Control Division.Okamura, A., and Cutkosky, M. 1999. Haptic exploration of fine surface features. Proc. of IEEE Int.Conf. on Robotics and Automation, pp. 2930–2936.Okamura, A. M., and Cutkosky, M. R. 2001. Feature detection for haptic exploration with roboticfingers. International Journal of Robotics Research 20, 12, 925–938.O’Sullivan, C., and Dingliana, J. 2001. Collisions and perception. ACM Trans. on Graphics 20, 3,pp. 151–168.O’Sullivan, C., Radach, R., and Collins, S. 1999. A model of collision perception for real-timeanimation. Computer Animation and Simulation, pp. 67–76.O’Sullivan, C., Dingliana, J., Giang, T., and Kaiser, M. K. 2003. Evaluating the visual fidelityof physically based animations. Proc. of ACM SIGGRAPH, 527–536.Otaduy, M. A., and Lin, M. C. 2003. CLODs: Dual hierarchies for multiresolution collision detection.Eurographics Symposium on Geometry Processing, 94–101.Otaduy, M. A., and Lin, M. C. 2003. Sensation preserving simplification for haptic rendering. ACMTrans. on Graphics (Proc. of ACM SIGGRAPH), 543–553.Otaduy, M. A., and Lin, M. C. 2005. Stable and responsive six-degree-of-freedom haptic manipulationusing implicit integration. Proc. of World Haptics Conference.Otaduy, M. A., Jain, N., Sud, A., and Lin, M. C. 2004. Haptic display of interaction betweentextured models. Proc. of IEEE Visualization, 297–304.Ouh-Young, M. 1990. Force Display in Molecular Docking. PhD thesis, University of North Carolina,Computer Science Department.Pai, D. K., and Reissel, L. M. 1997. Haptic interaction with multiresolution image curves. Computerand Graphics 21, 405–411.Pai, D. K., van den Doel, K., James, D. L., Lang, J., Lloyd, J. E., Richmond, J. L., and Yau,S. H. 2001. Scanning physical interaction behavior of 3d objects. Computer Graphics (Proc. of ACMSIGGRAPH).Quinlan, S. 1994. Efficient distance computation between non-convex objects. In Proceedings of InternationalConference on Robotics and Automation, 3324–3329.Redon, S., Kheddar, A., and Coquillart, S. 2002. Fast continuous collision detection between rigidbodies. Proc. of Eurographics (Computer Graphics Forum).A31

REFERENCES 30Renz, M., Preusche, C., Pötke, M., Kriegel, H.-P., and Hirzinger, G. 2001. Stable hapticinteraction with virtual environments using an adapted voxmap-pointshell algorithm. EurohapticsConference.Rock, I., and Victor, J. 1964. Vision and touch: An experimentally created conflict between the twosenses. Science 143, pp. 594–596.Ruspini, D., and Khatib, O. 2000. A framework for multi-contact multi-body dynamic simulation andhaptic display. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems.Ruspini, D., Kolarov, K., and Khatib, O. 1997. The haptic display of complex graphical environments.Proc. of ACM SIGGRAPH, 345–352.Salcudean, S. E., and Vlaar, T. D. 1994. On the emulation of stiff walls and static friction with amagnetically levitated input/output device. Proc. of ASME Haptic Interfaces for Virtual Environmentand Teleoperator Systems.Salisbury, K., Brock, D., Massie, T., Swarup, N., and Zilles, C. 1995. Haptic rendering:Programming touch interaction with virtual objects. In 1995 Symposium on Interactive 3D Graphics,P. Hanrahan and J. Winget, Eds., ACM SIGGRAPH, 123–130. ISBN 0-89791-736-7.Seidel, R. 1990. Linear programming and convex hulls made easy. In Proc. 6th Ann. ACM Conf. onComputational Geometry, 211–215.Shimoga, K. 1992. Finger force and touch feedback issues in dextrous manipulation. NASA-CIRSSEInternational Conference on Inetelligent Robotic Systems for Space Exploration.Siira, J., and Pai, D. K. 1996. Haptic textures – a stochastic approach. Proc. of IEEE InternationalConference on Robotics and Automation, 557–562.Slater, M., and Usoh, M. 1993. An experimental exploration of presence in virtual environments.Tech. Rep. 689, Department of Computer Science, University College London.Spence, C., Pavani, F., and Driver, J. 2000. Crossmodal links between vision and touch in covertendogenous spatial attention. Journal of Experimental Psychology: Human Perception and Performance26, pp. 1298–1319.Stewart, D. E., and Trinkle, J. C. 1996. An implicit time-stepping scheme for rigid body dynamicswith inelastic collisions and coulomb friction. International Journal of Numerical Methods in Engineering39, 2673–2691.Stewart, D. E., and Trinkle, J. C. 2000. An implicit time-stepping scheme for rigid body dynamicswith coulomb friction. IEEE International Conference on Robotics and Automation, 162–169.Stollnitz, E., DeRose, T., and Salesin, D. 1996. Wavelets for Computer Graphics: Theory andApplications. Morgan-Kaufmann.Sutherland, I. 1965. The ultimate display. Proc. of IFIP, 506–508.Taylor, R. M., Robinett, W., Chii, V., Brooks, F., and Wright, W. 1993. The nanomanipulator:A virtual-reality interface for a scanning tunneling microscope. In Proc. of ACM SIGGRAPH, 127–134.Thompson, T. V., Johnson, D., and Cohen, E. 1997. Direct haptic rendering of sculptured models.Proc. of ACM Symposium on Interactive 3D Graphics, pp. 167–176.A32

REFERENCES 31Unger, B. J., Nicolaidis, A., Berkelman, P. J., Thompson, A., Lederman, S. J., Klatzky,R. L., and Hollis, R. L. 2002. Virtual peg-in-hole performance using a 6-dof magnetic levitationhaptic device: Comparison with real forces and with visual guidance alone. Proc. of Haptics Symposium,263–270.van den Bergen, G. 2001. Proximity queries and penetration depth computation on 3d game objects.Game Developers Conference.Wan, M., and McNeely, W. A. 2003. Quasi-static approximation for 6 degrees-of-freedom hapticrendering. Proc. of IEEE Visualization, 257–262.Wu, D. 2000. Penalty methods for contact resolution. Game Developers Conference.Zilles, C., and Salisbury, K. 1995. A constraint-based god object method for haptics display. In Proc.of IEEE/RSJ Int. Conf. on Intelligent Robotics and Systems.Zorin, D., Schröder, P., and Sweldens, W. 1997. Interactive multiresolution mesh editing. Proc.of ACM SIGGRAPH, 259–268.A33

AFramework for Fast and Accurate Collision Detectionfor Haptic InteractionArthur Gregory Ming C. Lin Stefan Gottschalk Russell TaylorDepartment of Computer ScienceUniversity of North CarolinaChapel Hill, NC 27599-3175fgregory,lin,stefan,taylorrg@cs.unc.eduAbstractWe present a framework for fast and accurate collisiondetection for haptic interaction with polygonalmodels. Given a model, we pre-compute a hybrid hierarchicalrepresentation, consisting of uniform grids andtrees of tight-tting oriented bounding box trees (OBB-Trees). At run time, we use hybrid hierarchical representationsand exploit frame-to-frame coherence forfast proximity queries. We describe a new overlap test,which is specialized for intersection of a line segmentwith an oriented bounding box for haptic simulation andtakes 6-36 operations excluding transformation costs.The algorithms have been implemented aspart of H-COLLIDE and interfaced with a PHANToM arm andits haptic toolkit, GHOST, and applied to a number ofmodels. As compared tothecommercial implementation,we are able to achieve up to 20 times speedup inour experiments and sustain update rates over 1000Hzon a 400MHz Pentium II.1 IntroductionVirtual environments require natural interaction betweeninteractive computer systems and users. Comparedto the presentation of visual and auditory information,methods for haptic display arenotaswelldeveloped. Haptic rendering as an augmentation to visualdisplay can improve perception and understandingboth of force elds and of world models populated inthe synthetic environments [6]. It allows users to reachinto virtual worlds with a sense of touch, so they canfeel and manipulate simulated objects.Haptic display is often rendered through what is essentiallya small robot arm, used in reverse. Such devicesare now commercially available for a variety ofcongurations (2D, 3D, 6D, specialized for laparoscopyor general-purpose). The system used in this work wasa 6DOF-in/3DOF-out SensAble Technologies PHAN-ToM arm.\Real-time" graphics applications have display updaterequirements somewhere between 20 and 30frames/second. In contrast, the update rate of hapticsimulations must be as high as 1000 updates/second inorder to maintain a stable system. This rate varies withthe spatial frequency and stiness of displayed forces,and with the speed of motion of the user. Also, theskin is sensitive to vibrations of greater than 500 Hz,so changes in force at even relatively high frequenciesare detectable [10].In order to create a sense of touch between the user'shand and a virtual object, contact or restoring forcesare generated to prevent penetration into this virtualobject. This is computed by rst detecting if a collisionor penetration has occurred, then determining the(projected) contact point on the object surface. Mostof the existing algorithms are only sucient to addressthe collision detection and contact determination problemsfor relatively small models consisting of only a fewthousand polygons or a few surfaces. Our ultimate goalis to be able to achieve smooth, realistic haptic interactionwith CAD models of high complexity (normallyconsisted of tens of thousands of primitives) for virtualprototyping applications. In addition, we are aiming atdesigning algorithms that are easily extensible to supporta wide range of force-feedback devices (including6 degree-of-freedom arms) and deformable surfaces.Main Contribution: In this paper we present aframework for fast and accurate collision detection forhaptic interaction. It consists of a number of algorithmsand a system specialized for computing contact(s)between the probe of the force-feedback deviceand objects in the virtual environment. To meettheA34

stringent performance requirements for haptic interaction,we use a hybrid approach that specializes manyearlier algorithms for this application. Our frameworkutilizes: Spatial Decomposition: It decomposes theworkspace into uniform grids or cells, implementedas a hash table to eciently deal with large storagerequirements. At runtime, the algorithm canquickly nd the cell containing the path swept outby the probe. Bounding Volume Hierarchy based onOBBTrees: An OBBTree is a bounding volumehierarchy [15] oftight-tting oriented boundingboxes (OBBs). For each cell consisting of a subsetof polygons of the virtual model, we pre-computean OBBTree. At run-time, most of the computationtime is spent in nding collisions betweenan OBBTree and the path swept out by the tip ofthe probe between two successive time steps. Tooptimize this query, wehave developed a very fastspecialized overlap test between a line segmentandan OBB, that takes as few as 6 operations and only36 arithmetic operations in the worst case, not includingthe cost of transformation. Frame-to-Frame Coherence: Typically, thereis little movement in the probe position betweensuccessive steps. The algorithm utilizes this coherenceby caching the contact information fromthe previous step to perform incremental computations.The algorithm pre-computes a hybrid hierarchy. Ourframework also allows the application program to selectonly a subset of the approaches listed above.Wehave successfully implemented all the algorithmsdescribed above, interfaced them with GHOST (a commercialhaptic library) [34] and used them to nd surfacecontact points between the probe of a PHANToMarm and large geometric models (composed of tensof thousands of polygons). Their performance variesbased on the geometric model, the conguration ofthe probe relative to the model, machine conguration(e.g. cache and memory size) and the combinationof techniques used by our system. The overall approachresults in a factor of 2 ; 20 speed improvementas compared to earlier algorithms and commercial implementations.For a number of models composed of5 000;80 000 polygons, our system is able to substaina KHz update rate on a 400M Hz PC.The results presented in this paper are specializedfor a point probe against 3D object collision detection.We conjecture that it can be extended to computeobject-object intersection for a six-degree-of-freedomhaptic device.Organization: The rest of the paper is organized inthe following manner. Section 2 provides a brief surveyof related research. Section 3 describes the systemarchitecture and algorithms used in the design of oursystem. We discuss the implementation issues in Section4, present our experimental results and comparetheir performance with a commercial implementationin Section 5.2 Related WorkCollision detection and contact determination are wellstudiedproblems in computer graphics, computationalgeometry, robotics and virtual environments. Due tolimited space, we refer the readers to [22, 17] for recentsurveys. In the ray-tracing literature, the problem ofcomputing fast intersections between a ray and a threedimensionalgeometric model has also been extensivelystudied [1]. Whileanumber of algorithms have beenproposed that make use of bounding volume hierarchies,spatial partitioning or frame-to-frame coherence,there is relatively little available on hybrid approachescombining two or more such techniques.Bounding Volume Hierarchies: Anumber ofalgorithms based on hierarchical representations havebeen proposed. The set of bounding volumes includespheres [18, 30], axis-aligned bounding boxes [5, 17],oriented bounding boxes [15, 4], approximation hierarchiesbased on S-bounds [7], spherical shells [21] andk-dop's [20]. In the close proximity scenarios, hierarchiesof oriented bounding boxes (OBBTrees) appearsuperior to many other bounding volumes [15].Spatial Partitioning Approaches: Some of thesimplest algorithms for collision detection are based onspatial decomposition techniques. These algorithmspartition the space into uniform or adaptive grids(i.e. volumetric approaches), octrees [33], k-D trees orBSP's [27]. To overcome the problem of large memoryrequirements for volumetric approaches, some authors[29] have proposed the use of hash tables.Utilizing Frame-to-Frame Coherence: In manysimulations, the objects move only a little between successiveframes. Many ecient algorithms that utilizeframe-to-frame coherence have been proposed for convexpolytopes [23, 8, 3]. Cohen et al. [9] have usedcoherence-based incremental sorting to detect possiblepairs of overlapping objects in large environments.Research in Haptic Rendering: Several techniqueshave been proposed for integrating force feedbackwith a complete real-time virtual environment toenhance the user's ability to perform interaction tasks2A35

[10, 11, 12, 24, 25, 28, 35]. The commercial haptictoolkit developed by SensAbleTechnologies, Inc. alsohas a collision detection library probably using BSP-Trees [32, 34]. Ruspini et al. [31]have presented a hapticinterface library \HL" that uses a multi-level controlsystem to eectively simulate contacts with virtual environments.It uses a bounding volume hierarchy basedon sphere-trees [30].Nahvi et al. [26] have designed a haptic display systemfor manipulating virtual mechanisms derived froma mechanical CAD design. It uses the Sarcos DexterousArm Master and Utah's Alpha 1 CAD system,with algorithmic support from a tracing algorithm anda minimum distance framework developed by Johnson,Cohen et al. [36, 19]. They utilized a variety ofalgorithmictoolkits from RAPID [15] to build an OBBTreefor each object, Gilbert's algorithm [14] to nd distancesbetween two OBB's and a tracing algorithm forparametric surfaces. Their system takes about 20;150milliseconds for models composed of 500 ; 23 000 triangleson an SGI Indigo2 and 4 milliseconds for modelscomposed of 3 parametric surfaces on a Motorola 68040microprocessors.Gibson [13] and Sobierajski [2] have proposed algorithmsfor object manipulation including haptic interactionwith volumetric objects and physically-realisticmodeling of object interactions.3 Fast Proximity Queries forHaptic InteractionIn this section, we describe the haptic system setupand algorithmic techniques that are an integral partof the collision detection system framework for hapticinteraction, H-COLLIDE.3.1 Haptic System ArchitectureDue to the stringent update requirements for realtimehaptic display, we run a special standalonehaptic server written with the VRPN library(http://www.cs.unc.edu/Research/nano/manual/vrpn)on a PC connected to the PHANToM. The client applicationruns on another machine, which istypicallythe host for graphical display. Through VRPN, theclient application sends the server the description of thescene to be haptically displayed, and the server sendsback information such as the position and orientationof the PHANToM probe. The client application canalso modify and transform the scene being displayedby the haptic server.3.2 Algorithm OverviewGiven the last and current positions of the PHANToMprobe, we need to determine if it has in fact passedthrough the object's surface, in order to display the appropriateforce. The probe movement is usually smalldue to the high haptic update rates. This implies thatwe onlyneedtocheck a relatively small volume of theworkspace for collision detection.Approaches using spatial partitioning seem to benatural candidates for such situations. For large andcomplex models, techniques based on uniform or adaptivegrids can be implemented more eciently usinghash tables. However, to achieve the desired speed,these approaches still have extremely high storage requirementseven when implemented using a hashingscheme.Despite its better t to the underlying geometry,thehierarchical bounding volume method based on OBB-Trees may end up traversing trees to great depths tolocate the exact contact points for large, complex models.To take advantage of each approach and to avoidsome deciency of each, we propose a hybrid technique.Hybrid Hierarchical Representation: Given avirtual environment containing several objects, eachcomposed of tens of thousands of polygons, the algorithmcomputes a hybrid hierarchical representation ofthe objects as part of the o-line pre-computation. Itrst partitions the entire virtual workspace into coarsegrainuniform grid cells. Then, for each grid cell containingsome primitives of the objects in the virtualworld, it computes the OBBTrees for that grid cell andstores the pointer to the associated OBBTrees using ahash table for constant-time proximity queries.Specialized Intersection Tests: The on-line computationof our collision detection system consists ofthree phases. In the rst phase, it identies \the regionof potential contacts" by determining which cellswere touched by the probe path, using the precomputedlook-up table. In the second phase, it traversesthe OBBTree(s) in that cell to determine if collisionshave occurred, using the specialized fast overlap testto be described later. In the third phase, if the linesegment intersects with an OBB in the leaf node, thenit computes the (projected) surface contact point(s)(SCP) using techniques similar to those in [34, 36].Frame-to-Frame Coherence: If in the previousframe the probe of the feedback device was in contactwith the surface of the model, we exploit frameto-framecoherence by rstchecking if the last intersectedtriangle is still in contact with the probe. If so,we cache this contact witness. Otherwise, we check forcollision using hybrid hierarchical representation of theobjects.3A36

3.3 H-COLLIDEH-COLLIDE, a framework for fast and accurate collisiondetection for haptic interaction, is designed basedon the hybrid hierarchical representation and the algorithmictechniques described above. Figure 1 showsthe system architecture of H-COLLIDE.Compute hybrid hierarchial representationInput last position and current position / SCPCheck contact witnessfalseFind segment’s bounding grid cell(s)Query cell’s OBBTree(s)Check potential triangles for intersectionreturn FALSE or intersection point / SCPtrueofflineonlineFigure 1. The System Architecture of H-COLLIDEAxis Theorem described in [15], but specialized for aline segment against an OBB.Specically, the candidate axes are the three boxface normals (which are aligned with the coordinateaxes) and their cross-products with the segment's directionvector. With each of these six candidate axes,we project both the box and the segment onto it, andtest whether the projection intervals overlap. If theprojections are disjoint for any of the six candidateaxes, then the segment and the box are disjoint. Otherwise,the segment and the box overlap.How are the projection intervals computed? Givena direction vector v of a line through the origin, and apoint p, let the point p 0 be the axial projection of p ontothe line. The value d p = v p=jvj is the signed distanceof p 0from the origin along the line. Now consider theline segment with midpoint m and endpoints m + wand m ; w. The half-length of the line segment isjwj.The image of the segment under axial projection is theinterval centered atand with half-lengthd s = v m=jvjL s = jw vj=jvj3.4 Overlap Test based on a Line Segmentagainst an OBBTreeFor haptic display using a point probe, we can specializethe algorithm based on OBBTrees by only testinga line segment (representing the path swept outby the probe device between two successive steps) andan OBBTree. (The original algorithm [15] usesaoverlaptest between a pair of OBBs and can take morethan 200 operations per test.) At run time, most ofthe computation is spent in nding collisions betweena line segment and an OBB. To optimize this query,we have developed a very fast overlap test between aline segment and an OBB, that takes as few as 6 operationsand only 36 arithmetic operations in the worstcase, not including the cost of transformation.At the rst glance, it is tempting to use sophisticatedand optimized line clipping algorithms. However, theline-OBB intersection problem for haptic interactionis a simpler one than line clipping and the environmentis dynamic and consisting of many OBBs. Nextwe'll describe this specialized overlap test between aline segment and an oriented bounding box for hapticrendering. Without loss of generality, we will choosethe coordinate system centered on and aligned withthe box { so the problem is transformed to an overlaptest between a segment and a centered axis-alignedbounding box. Our overlap test uses the Separating-Given a box centered at the origin, the image of thebox under axial projection is an interval with midpointat the origin.Furthermore, if the box has thicknesses 2t x 2t y and2t z along the orthogonal unit directions u x u y and u z ,the half-length of the interval is given byL b = jt x v u x =jvjj + jt y v u y =jvjj + jt z v u z =jvjjWith the intervals so expressed, the axis v is a separatingaxis if and only if (see Figure 2)jd s j >L b + L sIf we assume that the box is axis-aligned, thenu x = [1 0 0] T u y = [0 1 0] T and u z = [0 0 1] T ,and the dot products with these vectors become simplecomponent selections. This simplies the box intervallength computation toL b = jt x v x j + jt y v y j + jt z v z jNow, recall that the candidate axis v is either a boxface normal, or a cross product of a face normal withthe line segment direction vector. Consider the formercase, when v is a box face normal, for example [1 0 0] T .In this case, the components v y and v z are zero, andthe component v x is one, and we are left withL b = t x4A37

m-wmThe half-length of the segment interval isL s = jw (w u y )j = ju y (w w)j = ju y 0j =0cL bL sd sm+wwhich iswhatwewould expect, since we are projectingthe segment onto a line orthogonal to it.Finally, the projection of the segments midpoint fallsatd s =(w u y ) m =(m w) u y = m z w x ; m x w zFigure 2. Overlap test between a line segment andan OBBThe projection of the line segment onto the x;axisis also simple:L s = jw x jSo, the test for the v =[1 0 0] T axis isjm x j >t x + jw x jThe tests for the candidate axes v =[0 1 0] T andv =[0 0 1] T have similar structure.The three cases where v is a cross product of w withone of the box faces are a little more complex. Recallthat in general,L b = jt x v u x j + jt y v u y j + jt z v u z jFor the sake of concreteness, we will choose v = w u y .Then this expression becomesL b = jt x (w u y )u x j+jt y (w u y ) u y j+jt z (w u y ) u z jApplication of the triple product identityyields(a b) c =(c a) bL b = jt x (u y u x )wj+jt y (u y u y ) wj+jt z (u y u z ) wjAll of these cross products simplify, because the u vectorsare mutually orthogonal, u x u y = u z u y u z =u x and u z u x = u y ,soL b = jt x (;u z ) wj + jt y (0) wj + jt z (u x ) wjAnd again, using the fact that u x =[1 0 0] T ,andsoforth,L b = t x jw z j + t z jw x jwhich isjustthey;component ofm w. The naltest isjm z w x ; m x w z j >t x jw z j + t z jw x jSimilar derivations are possible for the cases v =w u x and v = w u z .Writing out the entire procedure, and precomputinga few common subexpressions, we have the followingpseudo code:let X = jw x jlet Y = jw y jlet Z = jw z jif jm x j >X+ t x return disjointif jm y j >Y + t y return disjointif jm z j >Z+ t z return disjointif jm y w z ; m z w y j >t y Z + t z Y return disjointif jm x w z ; m z w x j >t x Z + t z X return disjointif jm x w y ; m y w x j >t x Y + t y X return disjointotherwise return overlapWhen a segment and an OBB are disjoint, the routineoften encounters an early exit and only one (ortwo) out of the six expressions is executed. Total operationcount for the worst case is: 9 absolute values, 6comparisons, 9 add and subtracts, 12 multiplies. Thisdoes not include the cost of transforming, i.e. 36 operations,the problem into a coordinate system centeredand aligned with the box.4 Implementation IssuesH-COLLIDE has been successfully implemented inC++. We have interfaced H-COLLIDE with GHOST,a commercial software developer's toolkit for hapticrendering, and used it to nd surface contact pointsbetween the probe of a PHANToM arm and large geometricmodels (composed of tens of thousands of polygons).Here we describe some of the implementationissues.5A38

4.1 Hashing SchemeClearly it is extremely inecient to allocate storage forall these cells, since a polygonal surface is most likelyto occupy avery small fraction of them. We use a hashtable to alleviate the storage problem. From each celllocation at (x y z) and a grid that has len cells in eachdimension, we can compute a unique key usingkey = x + y len + z len 2 .In order to avoid hashing too many cells with samepattern into the same table location we compute theactual location for a grid cell in the hash table withT ableLoc = random(key)%T ableLength.Should the table have too many cells in one tablelocation, we can simply grow the table. Hence, it ispossible to determine which triangles we need to checkin constant time and the amount of storage requiredis a constant factor (based on the grid grain) of thesurface area of the object we want to \feel".Determining the optimal grid grain is a non-trivialproblem. Please refer to [16] for a detailed retreatmentand a possible analytical solution to this problem.We simply set the grain of the grids to be the averagelength of all edges. If the model has a very irregulartriangulation it is very possible that there could be alarge number of small triangles in a single grid cell.Querying an OBBTree takes O(logn) time, wheren is the number of triangles in the tree. During theo-line computation, we can ensure that n is a smallnumber compared to the total number of triangles inthe model thus the overall running time of our hybridapproach should be constant.4.2 User OptionsSince the hybrid approach used in H-COLLIDE has ahigher storage requirement than either the individualtechnique alone, the system also allows the user to selecta subset of the techniques, such as the algorithmpurely based on OBBTrees, to opt for better performanceon a machine with less memory.5 System PerformanceFor comparison, we have implemented adaptive grids,our hybrid approach and an algorithm using only OBB-Trees and the specialized overlap test described in Section3.4. Wehave applied them to a wide range of modelsof varying sizes. (Due to the page limit, we invitethe readers to view the Color Plates of these models athttp://www.cs.unc.edu/~geom/HCollide/model.pdf.)Their performance varies based on the models, the con-guration of the probe relative to the model, machineconguration (e.g. cache and memory size) and thecombination of techniques used by our system. Ourhybrid approach results in a factor of 2-20 speed improvementas compared to a native GHOST method.For a number of models composed of 5 000 ; 80 000polygons, our system is able to compute all the contactsand response at rates higher than 1000 Hz on a400MHz PC.5.1 Obtaining Test DataWe rst obtained the test data set by deriving aclass from the triangle mesh primitive which comeswith SensAble Technologies' GHOST library, version2.0 beta. This records the start and the endpoint ofeach segment used for collision detection during a realforce-feedback session with a 3-DOF PHANToM arm.We then implemented the three techniques mentionedabove tointerface with GHOST for comparison with anative GHOST method, and timed the collision detectionroutines for the dierent libraries using the datafrom the test set. The test set for each of these modelscontains 30,000 readings.The distinction between a collision and an intersectionshown in the tables is particular to GHOST'shaptic rendering. Each haptic update cycle contains a\collision" test to see if the line segment from the lastposition of the PHANToM probe to its current positionhas intersected any of the geometry in the hapticscene. If there has been a collision, then the intersectedprimitive suggests a surface contact point forthe PHANToM probe to move towards. In this caseit is now necessary to perform an \intersection" testto determine if the line segment from the last positionof the PHANToM probe to the suggested surface contactpoint intersects any of the geometry in the scene(including the primitive with which therewas a \collision").The timings (in milliseconds) shown in Tables 1-5were obtained by replaying the test data set on a 4processor 400 MHz PC, with 1 GB of physical memory.Each timing was obtained using only one processor.For comparison, we ran the same suite of tests on asingle processor 300 MHz Pentium Pro with 128 MBmemory. Thehybrid approach appeared to be the mostfavorable as well.5.2 Comparison between AlgorithmsSince the algorithms run on a real-time system, we arenot only interested in the average performance, butalso the worst case performance. Tables 1-5 show the6A39

Method Hash Grid Hybrid OBBTree GHOSTAve Col. Hit 0.0122 0.00883 0.0120 0.0917Worst Col. Hit 0.157 0.171 0.0800 0.711Ave Col. Miss 0.00964 0.00789 0.00856 0.0217Worst Col. Miss 0.0753 0.0583 0.0683 0.663Ave Int. Hit 0.0434 0.0467 0.0459 0.0668Worst Int. Hit 0.108 0.102 0.0793 0.100Ave Int. Miss 0.0330 0.0226 0.0261 0.0245Worst Int. Miss 0.105 0.141 0.0890 0.364Ave. Query 0.019 0.014 0.017 0.048Table 1. Timings in msecs for Man Symbol, 5K trisMethod Hash Grid Hybrid OBBTree GHOSTAve Col. Hit 0.0115 0.0185 0.0109 0.131Worst Col. Hit 0.142 0.213 0.138 0.622Ave Col. Miss 0.0104 0.00846 0.0101 0.0176Worst Col. Miss 0.0800 0.0603 0.0813 0.396Ave Int. Hit 0.0583 0.0568 0.0652 0.0653Worst Int. Hit 0.278 0.200 0.125 0.233Ave Int. Miss 0.0446 0.0237 0.0349 0.0322Worst Int. Miss 0.152 0.173 0.111 0.287Ave. Query 0.030 0.025 0.028 0.070Table 2. Timings in msecs for Man with Hat, 7K trisMethod Hash Grid Hybrid OBBTree GHOSTAve Col. Hit 0.0138 0.0101 0.0134 0.332Worst Col. Hit 0.125 0.168 0.0663 0.724Ave Col. Miss 0.00739 0.00508 0.00422 0.0109Worst Col. Miss 0.0347 0.0377 0.0613 0.210Ave Int. Hit 0.0428 0.0386 0.0447 0.0851Worst Int. Hit 0.0877 0.102 0.0690 0.175Ave Int. Miss 0.0268 0.0197 0.0213 0.0545Worst Int. Miss 0.0757 0.0697 0.0587 0.284Ave. Query 0.022 0.016 0.039 0.18Table 3. Timings in msecs for Nano Surface, 12K trisMethod Hash Grid Hybrid OBBTree GHOSTAve Col. Hit 0.0113 0.00995 0.0125 0.104Worst Col. Hit 0.136 0.132 0.177 0.495Ave Col. Miss 0.0133 0.00731 0.0189 0.0280Worst Col. Miss 0.128 0.0730 0.137 0.641Ave Int. Hit 0.0566 0.0374 0.609 0.0671Worst Int. Hit 0.145 0.105 0.170 0.293Ave Int. Miss 0.0523 0.0225 0.0452 0.0423Worst Int. Miss 0.132 0.133 0.167 0.556Ave. Query 0.027 0.014 0.028 0.048Table 4. Timings in msecs for Bronco, 18K trisMethod Hash Grid Hybrid OBBTree GHOSTAve Col. Hit 0.0232 0.0204 0.0163 1.33Worst Col. Hit 0.545 0.198 0.100 5.37Ave Col. Miss 0.00896 0.00405 0.00683 0.160Worst Col. Miss 0.237 0.139 0.121 3.15Ave Int. Hit 0.228 0.0659 0.0704 0.509Worst Int. Hit 0.104 0.138 0.103 1.952Ave Int. Miss 0.258 0.0279 0.0256 0.229Worst Int. Miss 0.0544 0.131 0.0977 3.28Ave. Query 0.030 0.016 0.016 0.320Table 5. Timings in msecs for Buttery, 79K tristimings in milliseconds obtained for both cases on eachmodel and each contact conguration.All our algorithms are able to perform collisionqueries at rates faster than the required 1000 Hz forceupdate rate for all models in the worst case. Althoughthe hybrid approach often outperforms the algorithmbased on OBBTrees, it is sometimes slightly slowerthan the alogrithm based on OBBTrees. We conjecturethat this behavior is due to the cache size of the CPU(independent of the memory size) and memory pagingalgorithm of the operating system. Among techniquesthat use hierarchical representations, cache access patternscan often have a dramatic impact on run timeperformance.The hybrid approach requires more memory and islikely to have a less cache-friendly memory access patternthan the algorithm purely based on OBBTrees,despite the fact that both were well within the realmof physical memory available to the machine. Furthermore,by partitioning polygons into groups using grids,the hybrid technique can enable real-time local surfacemodication.The adaptive grids-hashing scheme, a commonlyused technique in ray-tracing, did not perform equallywell in all cases. Once again, our hypothesis is thatits inferior worst case behavior is due to its cache accesspatterns, in addition to its storage requirements.We believe the native GHOST method uses an algorithmbased BSP trees. While it is competitive for thesmaller model sizes, its performance fails to scale upfor larger models. Our hybrid approach and our algorithmpurely based on OBBTrees and the specializedoverlap test appear to be relatively unaected by themodel complexity.6 ConclusionWe have presented a framework, H-COLLIDE, thatconsists of a suite of algorithms and a system implementationfor fast and accurate collision detectionfor haptic interaction with polygonal models at rateshigher than 1000Hz on a desk-top PC. This frameworkmay be extended for supporting 6-DOF haptic devicesto perform collision tests between a pair of 3D objectsand exible surfaces that may deform due to manipulation.In addition, it can be combined with the tracingalgorithm [36] to handle complex sculptured modelsmore eciently, by using their control points.Acknowledgements: We are grateful toNational Science Foundation, National Institute ofHealth National Center for Research Resources and IntelCorporation for their support and the reviewers fortheir comments.7A40

References[1] J. Arvo and D. Kirk. A survey of ray tracing acceleration techniques.In An Introduction to Ray Tracing, pages 201{262,1989.[2] R. S. Avila and L. M. Sobierajski. A haptic interaction methodfor volume visualization. Proceedings of Visualization'96,pages 197{204, 1996.[3] D. Bara. Curved surfaces and coherence for non-penetratingrigid body simulation. ACM Computer Graphics, 24(4):19{28,1990.[4] G. Barequet, B. Chazelle, L. Guibas, J. Mitchell, and A. Tal.Boxtree: A hierarchical representation of surfaces in 3d. InProc. of Eurographics'96, 1996.[5] N. Beckmann, H. Kriegel, R. Schneider, and B. Seeger. Ther*-tree: An ecient and robust access method for points andrectangles. Proc. SIGMOD Conf. on Management of Data,pages 322{331, 1990.[6] Frederick P. Brooks, Jr., Ming Ouh-Young, James J. Batter,and P. Jerome Kilpatrick. Project GROPE | Haptic displaysfor scientic visualization. In Forest Baskett, editor, ComputerGraphics (SIGGRAPH '90 Proceedings), volume 24, pages177{185, August 1990.[7] S. Cameron. Approximation hierarchies and s-bounds. InProceedings. Symposium on Solid Modeling Foundations andCAD/CAM Applications, pages 129{137, Austin, TX, 1991.[8] Stephen Cameron. A comparison of two fast algorithms for computingthe distance between convex polyhedra. IEEE Transactionson Robotics and Automation, 13(6):915{920, December1996.[9] J. Cohen, M. Lin, D. Manocha, and M. Ponamgi. I-collide:An interactive and exact collision detection system for largescaleenvironments. In Proc. of ACM Interactive 3D GraphicsConference, pages 189{196, 1995.[10] J. E. Colgate and J. M. Brown. Factors aecting the z-width of ahaptic display. IEEE Conference onRobotics and Automation,pages 3205{3210, 1994.[11] J. E. Colgate and et al. Issues in the haptic display of tooluse. Proceedings of the ASME Haptic Interfaces for VirtualEnvironment and Teleoperator Systems, pages 140{144, 1994.[12] M. Finch, M. Falvo, V. L. Chi, S. Washburn, R. M. Taylor,and R. Superne. Surface modication tools in a virtual environmentinterface to a scanning probe microscope. In PatHanrahan and Jim Winget, editors, 1995 SymposiumonInteractive3D Graphics, pages 13{18. ACM SIGGRAPH, April1995.[13] S. Gibson. Beyond volume rendering: Visualization, haptic exploration,an d physical modeling of element-based objects. InProc. Eurographics workshop on Visualization in ScienticComputing, pages 10{24, 1995.[14] E. G. Gilbert, D. W. Johnson, and S. S. Keerthi. A fast procedurefor computing the distance between objects in threedimensionalspace. IEEE J. Robotics and Automation, volRA-4:193{203, 1988.[15] S. Gottschalk, M. Lin, and D. Manocha. Obb-tree: A hierarchicalstructure for rapid interference detection. In Proc. of ACMSiggraph'96, pages 171{180, 1996.[16] A. Gregory, M. Lin, S. Gottschalk, and R. Taylor. H-collide:A framework for fast and accurate collision detection for hapticinteraction. Technical report, Department of Computer Science,University of North Carolina, 1998.[17] M. Held, J.T. Klosowski, and J.S.B. Mitchell. Evaluation ofcollision detection methods for virtual reality y-throughs. InCanadian Conference on Computational Geometry, 1995.[18] P. M. Hubbard. Interactive collision detection. In Proceedingsof IEEE Symposium on Research Frontiers in Virtual Reality,October 1993.[19] D. Johnson and E. Cohen. A framework for ecient minimumdistance computation. IEEE Conference onRobotics and Automation,pages 3678{3683, 1998.[20] J. Klosowski, M. Held, J.S.B. Mitchell, H. Sowizral, andK. Zikan. Ecient collision detection using bounding volumehierarchies of k-dops. In Siggraph'96 Visual Proceedings, page151, 1996.[21] S. Krishnan, A. Pattekar, M. Lin, and D. Manocha. Sphericalshell: A higher order bounding volume for fast proximityqueries. In Proc. of Third International Workshop on AlgorithmicFoundations of Robotics, pages 122{136, 1998.[22] M. Lin and S. Gottschalk. Collision detection between geometricmodels: A survey. InProc. of IMA Conference onMathematicsof Surfaces, 1998.[23] M.C. Lin and John F. Canny. Ecient algorithms for incrementaldistance computation. In IEEE ConferenceonRobotics andAutomation, pages 1008{1014, 1991.[24] William Mark, Scott Randolph, Mark Finch, James Van Verth,and Russell M. Taylor II. Adding force feedback to graphicssystems: Issues and solutions. In Holly Rushmeier, editor,SIGGRAPH 96 Conference Proceedings, Annual ConferenceSeries, pages 447{452, 1996.[25] T. M. Massie and J. K. Salisbury. The phantom haptic interface:A device for probing virtual objects. Proc. of ASME HpaticInterfaces for Virtual Environment and Teleoperator Systems,1:295{301, 1994.[26] A. Nahvi, D. Nelson, J. Hollerbach, and D. Johnson. Haptic manipulationof virtual mechanisms from mechanical cad designs.In Proc. of 1998 Conference onRobotics and Automation,pages 375{380, 1998.[27] B. Naylor, J. Amanatides, and W. Thibault. Merging bsp treesyield polyhedral modeling results. In Proc. of ACM Siggraph,pages 115{124, 1990.[28] M. Ouh-Young. Force Display in Molecular Docking. PhDthesis, University of North Carolina, Computer Science Department,1990.[29] M. H. Overmars. Point location in fat subdivisions. Inform.Proc. Lett., 44:261{265, 1992.[30] S. Quinlan. Ecient distance computation between nonconvexobjects. In Proceedings of International Conferenceon Robotics and Automation, pages 3324{3329, 1994.[31] D.C. Ruspini, K. Kolarov, and O. Khatib. The haptic display ofcomplex graphical environments. Proc. of ACM SIGGRAPH,pages 345{352, 1997.[32] K. Salisbury, D. Brock, T. Massie, N Swarup, and C. Zilles.Haptic rendering: Programming touch interaction with virtualobjects. Proc. of 1995 ACM Symposium on Interactive 3DGraphics, pages 123{130, 1995.[33] H. Samet. Spatial Data Structures: Quadtree, Octrees andOther Hierarchical Methods. Addison Wesley, 1989.[34] Inc. SensAble Technologies. ghost TM : Sofware developer'stoolkit. Programmer's Guide, 1997.[35] R. M. Taylor, W. Robinett, V.L. Chii, F. Brooks, andW. Wright. The nanomanipulator: A virtual-reality interfacefor a scanning tunneling microscope. In Proc. of ACM Siggraph,pages 127{134, 1993.[36] T.V. Thompson, D. Johnson, and E. Cohen. Direct haptic renderingof sculptured models. Proc. of ACM Interactive 3DGraphics, pages 167{176, 1997.8A41

Six Degree-of-Freedom Haptic Rendering Using Voxel SamplingWilliam A. McNeely Kevin D. Puterbaugh James J. TroyThe Boeing Company *AbstractA simple, fast, and approximate voxel-based approach to 6-DOF haptic rendering is presented. It can reliably sustain a 1000Hz haptic refresh rate without resorting to asynchronous physicsand haptic rendering loops. It enables the manipulation of a modestlycomplex rigid object within an arbitrarily complex environmentof static rigid objects. It renders a short-range force fieldsurrounding the static objects, which repels the manipulated objectand strives to maintain a voxel-scale minimum separation distancethat is known to preclude exact surface interpenetration. Force discontinuitiesarising from the use of a simple penalty force modelare mitigated by a dynamic simulation based on virtual coupling.A generalization of octree improves voxel memory efficiency. In apreliminary implementation, a commercially available 6-DOFhaptic prototype device is driven at a constant 1000 Hz hapticrefresh rate from one dedicated haptic processor, with a separateprocessor for graphics. This system yields stable and convincingforce feedback for a wide range of user controlled motion inside alarge, complex virtual environment, with very few surface interpenetrationevents. This level of performance appears suited toapplications such as certain maintenance and assembly task simulationsthat can tolerate voxel-scale minimum separation distances.CR Categories and Subject Descriptors: H.5.2 [User Interfaces]:Haptic I/O, I.3.5 [Computational Geometry and ObjectModeling]: Physically Based Modeling.Additional Keywords: force feedback, voxel representations,virtual environments.1. INTRODUCTIONThe problem of simulating real-world engineering tasks — forexample, objectives like design-for-assembly and design-for-maintenance— has been exacerbated by the modern transition fromphysical mockup to virtual mockup. Physical mockup providesnatural surface constraints that prevent tools and parts from interpenetrating,whereas virtual mockup requires the user to satisfysuch constraints by receiving collision cues and making appropriatebody postural adjustments, which is usually tedious and mayyield dubious results. In order to emulate the natural surface constraintsatisfaction of physical mockup, one must introduce force* Bill McNeely, The Boeing Company, P.O. Box 3707, M/S 7L-43, Seattle, WA98124, (bill.mcneely@boeing.com)Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copiesare not made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. To copyotherwise, to republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.SIGGRAPH 99, Los Angeles, CA USACopyright ACM 1999 0-201-48560-5/99/08 . . . $5.00feedback into virtual mockup. Doing so shifts the burden of physicalconstraint satisfaction onto a haptic subsystem, and the userbecomes free to concentrate on higher-level problems such as pathplanning and engineering rule satisfaction.Tool and part manipulation inherently requires six degree-offreedom(6-DOF) haptics, since extended objects are free to movein three translational and three rotational directions. Affordablehigh-bandwidth 6-DOF devices are becoming available, but 6-DOF haptic rendering remains an outstanding problem. It is considerablymore difficult than 3-DOF point-contact haptic rendering.One can compare haptics with collision detection, since theyshare some technical similarity. Real-time collision detection is achallenging problem [13], but 6-DOF haptics adds stringent newrequirements such as:• Detect all surface contact (or proximity, for a force field),instead of stopping at the first evidence of it.• Calculate a reaction force and torque at every point or extendedregion of contact/proximity.• Reliably maintain a 1000 Hz refresh rate, independent of positionand orientation of the manipulated object.• Control geometry driven haptic instabilities, such as forcing anobject into a narrow wedge-shaped cavity.To address the needs of our targeted engineering applications,we adopt the following additional goals:• Minimize the interpenetration of exact surface representations.• Handle complex static scenes, e.g., those containing severalhundred thousand triangles, with reasonable memory efficiency.• The haptic rendering algorithm should parallelize easily.Furthermore, we accept the limitation of voxel-scale accuracy. Forexample, a common engineering rule is to design at least 0.5 inchclearance into part removal paths, whenever possible, in order toaccommodate tool access and human grasping and to serve as acushion against assembly tolerance buildup.We describe an approach that formally meets most of theserequirements. It demonstrates the ability to drive a commerciallyavailable 6-DOF prototype device at a reliable 1000 Hz hapticrefresh rate without the aid of asynchronous physics and hapticrendering loops. It supports the manipulation of a single rigidobject within an arbitrarily rich environment of static rigid objectsby rendering a half-voxel-deep force field that surrounds the staticobjects and serves to block potential interpenetration of the exactsurface representations, as described in section 4.2. Given a predeterminedspatial accuracy (i.e., voxel size), rendering performancedepends linearly on the total exposed surface area of the manipulatedobject. There is also a relatively minor dependence on theinstantaneous amount of contact/proximity, with a worst-case performance(e.g., maximum contact/proximity) of about half that ofthe best-case performance.401A42

Our approach is distinguished primarily by its high haptic renderingspeed, which is derived primarily from:• A simple penalty force scheme called the tangent-plane forcemodel, explained in section 3.• A fixed-depth voxel tree, explained in section 4.3.• A voxel map that collectively represents all static objects,explained in section 4.4.Although the simplicity of our force model is critically importantto performance, it is so simple that it generates force magnitudediscontinuities (but not force direction discontinuities),especially under sliding motion. In 3-DOF point-contact haptics,force discontinuities can be devastating to force quality and stability,but under our 6-DOF approach there is a stochastic effect thatlessens their impact. However, it proved necessary to introducevarious measures to explicitly enhance force quality and stability,such as:• A single-body dynamic model based on virtual coupling• Pre-contact braking forcesAll such measures are explained in section 5.Data storage is often a secondary consideration in haptics work,because it is tempting to trade memory efficiency for higher performance.However, voxels are so relatively inefficient as geometricmodeling elements that we improve their memory efficiency bygeneralizing the octree method, as explained in section 4.3.2. PREVIOUS WORKAlthough largely the result of unpublished work, there arenumerous examples of 6-DOF haptic rendering for scenarios containinga very limited number of geometrically well behaved virtualobjects, for example [6,7,24]. Our approach differs from thiswork primarily in its ability to render considerably more complex6-DOF scenarios with no formal constraints on object shape,although at reduced accuracy.Our approach includes a collision detection technique based onprobing a voxelized environment with surface point samples.Voxel-based methods have been applied to non-haptic collisiondetection [12,15,16] and to 3-DOF haptics [3,18]. Sclaroff andPentland [22] apply surface point sampling to implicit surfaces.Intermediate representations for haptics were suggested by Adachiet al. [1], and have been subsequently elaborated [17]. Thisinvolves using a simple haptics proxy that approximates the exactscene and is simple enough to update the forces at the requiredhigh refresh rate, while a slower but more exact collision detectionand/or dynamic simulation runs asynchronously and updates theproxy’s parameters. Our work differs by tightly integrating collisiondetection, the force model, and the dynamic model into a singleloop that updates forces directly at 1000 Hz.There has been much work in multibody dynamic simulation forphysically based modeling, for example [4,23]. Mirtich and Canny[19] track the contacts found from an iterative collision detectionmethod and use this information to generate constant-sizeimpulses. In general, such work is characterized by its emphasison accuracy over rendering performance, and consequently itrelies on methodology such as exact-surface collision detectionand simultaneous surface constraint satisfaction, which currentlyfall far short of 6-DOF haptics performance requirements.Our dynamic model adopts the practice of using an artificialcoupling between the haptic display and virtual environment, asoriginally proposed by Colgate et al. [10] and recently elaboratedby Adams and Hannaford [2]. We also adopt a version of the “godobject” concept suggested by Zilles and Salisbury [25] and others[21], generalized to 6-DOF and modified to use penalty forces thatonly approximately satisfy surface constraints. In addition, we usethe concept of pre-contact braking force suggested by Clover [9].Hierarchical techniques, such as employed by Gottschalk [13],can be used to alleviate convex-hull bounding box limitations forobjects in very close proximity by recursively generating a tree ofbounding volumes around finer features of the object. While thistechnique speeds collision detection, it also introduces indeterminacyin the cycle rate due to the varying cost of traversing the treestructure to an unknown depth to check each colliding polygonagainst object polygons. Cycle-rate should not only be fast butshould also have a rate that is as constant as possible.Temporal and spatial coherence can also be exploited [4,5,8] byassuming that objects move only slightly within each time step,thus allowing extrapolation from the previous state of the system.The number of polygon tests carried out at each time step is effectivelyreduced, increasing cycle-rate at the cost of introducingindeterminacy. With certain configurations or motions of objects,however, there are often noticeable drops in performance — a situationwhich is unacceptable in a real-time simulation.3. TANGENT-PLANE FORCE MODELIn our tangent-plane force model, dynamic objects are representedby a set of surface point samples, plus associated inwardpointing surface normals, collectively called a point shell. Duringeach haptic update the dynamic object’s motion transformation isapplied to every point of the point shell. The environment of staticobjects is collectively represented by a single spatial occupancymap called a voxmap, which is illustrated in Figure 1. Each hapticallyrendered frame involves sampling the voxmap at every pointof the point shell.OriginalObjectsFigure 1. Voxmap colliding with point shell.Point Shelland NormalsVoxmapWhen a point interpenetrates a voxel (assumed for now to be asurface voxel) as shown in Figure 2, a depth of interpenetration iscalculated as the distance d from the point to a plane within thevoxel called the tangent plane.The tangent plane is dynamically constructed to pass throughthe voxel’s center point and to have the same normal as the point’sassociated normal. If the point has not penetrated below that plane(i.e., closer to the interior of the static object), then d is zero. Forceis simply proportional to d by Hooke’s law ( F = K ff d ). We callK ff the “force field stiffness,” since the voxel represents a half-402A43

voxel-deep force field. The net force and torque acting on thedynamic object is obtained as the sum of all force/torque contributionsfrom such point-voxel intersections.dForce Vector AlongPoint NormalStaticSurfaceFigure 2. Tangent-plane force model.TangentPlanePointShellThe tangent-plane force model was inspired by the fact that thesurfaces of contacting objects are tangent at an osculation point. Itis important that the force takes its direction from a precomputedsurface normal of the dynamic object. This proves to be considerablyfaster than the common practice of dynamically computing itfrom the static object’s surface, or in the case of a force field,dynamically taking the gradient of a potential field.One can see that this simple model has discontinuities in forcemagnitude when a point crosses a voxel boundary, for example,under sliding motion. Section 5 describes how discontinuities canbe mitigated for haptic purposes.4. VOXEL DATA STRUCTURESThis section outlines the creation and usage of voxel-based datastructures that are required under our approach. Exact (polygonal)surface penetration and memory usage will also be discussed.4.1 Voxmap and Point ShellOne begins by selecting a global voxel size, s, that meets the virtualscenario’s requirements for accuracy and performance. Theperformance aspect is that the force model requires traversing a setof point samples, and s determines the number of such points.Consider a solid object such as the teapot in Figure 3(a). It partitionsspace into regions of free space, object surface, and objectinterior. Now tile this space into a volume occupancy map, or voxmap,as in Figure 3(b). The collection of center points of all surfacevoxels constitutes the point shell needed by the tangent-planeforce model, as in Figure 3(c).(a) (b) (c)Figure 3. Teapot: (a) polygonal model, (b) voxel model, (c)point shell model.This method for creating the point shell is not optimal, but it isconvenient. Its accuracy may be improved by choosing points thatlie on the exact geometrical representation.Each voxel is allocated two bits of memory that designate it as afree space, interior, surface, or proximity voxel. The 2-bit voxeltypes are defined in Table 1 and illustrated by an example inFigure 4.A neighbor voxel is defined as sharing a vertex, edge, or facewith the subject voxel. Each voxel has 26 neighbors. It is importantthat each static object be voxelized in its final position and orientationin the world frame, because such transformations cause itsvoxelized representation to change shape slightly.Table 1. Voxel types (2-bit)Value Voxel type Description0 Free space Encloses only free-space volumes1 Interior Encloses only interior volumes2 Surface Encloses a mix of free-space, surface,and interior volumes3 Proximity Free-space neighbor of a surfacevoxelFigure 4. Assignment of 2-bit voxel values.By the nature of 3D scan conversion, voxmaps are insensitive tosurface imperfections such as gaps or cracks that are smaller thanthe voxel width. However, identifying the interior of a voxmap canbe difficult. We adopt the practice of (1) scan-converting to createsurface voxels, (2) identifying free-space voxels by propagatingthe voxelized walls of the object’s bounding box inward until surfacevoxels are encountered, and (3) declaring all other voxels tobe interior voxels. This ensures that objects with open surfaceswill be voxelized instead of “leaking” and filling all voxels.4.2 Avoiding Exact Surface InterpenetrationIn the tangent-plane force model shown in Figure 2, the exactsurfaces of colliding objects are allowed to interpenetrate byvoxel-scale distances during a point-voxel intersection. While thismay be acceptable for some applications, we seek instead to precludeexact-surface interpenetration. We do this by offsetting theforce field outward away from the surface by two voxel layers, asshown in Figure 5. (In this figure, the rotated boxes represent thesurface voxels associated with the points of a pointshell, viewed assurface bounding volumes.) The offset force layer then serves tomaintain a minimum object separation that provably precludesexact-surface interpenetration.Offset LayersExact Surface0000{2113333Figure 5. Criterion for exact-surface interpenetration.3222OK22111111Exact SurfaceBADForce LayerSurface Layer403A44

The voxel legend described by Table 1 and Figure 4 is correspondinglyredefined so that “surface” and “value 2” now refer tothe offset force-layer voxels instead of geometric surface voxels,and similarly for the other voxel types. (Offset proximity voxelsand free-space voxels are omitted from Figure 5, but they wouldoccupy additional layers at the top of the figure.)Force-layer offsetting is implemented as a final step of voxelization,in which the geometric surface voxel layer is grown outwardby a process of promoting proximity voxels to surface values anddemoting original surface voxels to interior values. This process isrepeated to achieve the desired two-layer offset. (If voxels wereallocated more than two bits, it would not be necessary to “recycle”voxel values in this manner, and there are other advantages towider voxels that we are beginning to explore.)Force-layer offsetting also serves to prevent any spike-like featurein the static object from generating a linear column of voxelsthat the point shell could completely fail to penetrate for certainorientations of the dynamic object. The force layer has no suchfeatures, because voxel values are propagated to 26 connectedneighbors during the offsetting process.4.3 Voxel TreeA natural next step is to impose an octree organization on thevoxels for the sake of memory efficiency and scalability. However,the need for a consistently fast haptic refresh rate is at odds withthe variability in the tree traversal time. To address this, we havedevised a hierarchy that represents a compromise between memoryefficiency and haptic rendering performance. It is a generalizationof octree with a tree depth that is limited to three levels,explained as follows.At each level of the tree, the cubical volume of space is dividedinto 2 3N sub-volumes, where N is a positive integer. (N is unity foran octree.) We have discovered that the most memory-efficientvalue for N may be at higher values, depending on the sparsenessof the geometry. Figure 6 illustrates a study of the total memoryconsumed by a 2 3N -tree as a function of N for geometry that is typicalto our work. It has a minimum at N=3, which might be called a512-tree.Memory, MB14070605040302010(Octree)1 2 3 4 5Exponent, NFigure 6. Memory usage of 2 3N tree as a function of N.We further limit tree depth by fixing both the minimum andmaximum dimensions of the bounding volumes in the tree. Theminimum dimension is the size of voxels at the leaf level, and themaximum dimension is given implicitly by creating only three levelsabove the leaf level. The minimum-size requirement means thatsmaller features may not be adequately represented, but we fundamentallyaccept a global accuracy limitation, analogous to thepractice of accepting a fixed tessellation error in polygonal surfacerepresentations. The maximum-size requirement impacts memoryefficiency and scalability, because one must cover all remainingspace with the largest-size bounding volumes. However, theseeffects are mitigated by the use of 2 3N -tree, since for a fixed numberof levels, higher values of N increase the dynamic range of thebounding volume dimensions.4.4 Merged Scene VoxmapOur approach is limited to the case of a single dynamic rigidobject interacting with an arbitrarily rich environment of staticrigid objects. If it were necessary to separately calculate the interactionforce for each of N static objects, then the computing burdenwould grow linearly with N. However, there is no inherentneed to separately compute such interactions on a pairwise basis.For example, there is no need to identify the type of a contactedobject in order to apply different material properties, since allstatic objects are treated as rigid. Furthermore, under our forcefieldapproach, objects are never actually contacted in the sense ofundergoing surface intersections. Therefore, we merge all staticobjectvoxel representations together as if they were a single staticobject, applying straightforward precedence rules to merged voxelvalues and recalculating a voxel tree for the voxmap.5. DYNAMIC MODELFor the dynamic model, we use an impedance approach, inwhich user motion is sensed and a force/torque pair is produced.We further adopt what is called the “virtual coupler” scheme,which connects the user’s haptic motions with the motions of thedynamic object through a virtual spring and damper. This is a wellknown method for enhancing haptic stability [2].To solve for the motion of the dynamic object, we perform anumerical integration of the Newton-Euler equation, using a constanttime step ∆t corresponding to the time between forceupdates, e.g., ∆t =1 msec for 1000 Hz haptic refresh rate. We alsomust assign a mass m to the dynamic object equal to the apparentmass for the dynamic object that we want to feel at the haptic handle(in addition to the haptic device’s intrinsic friction and inertia,and assuming that its forces are not yet saturated). The net forceand torque on the dynamic object is the sum of contributions fromthe spring-damper system, explained in section 5.1; stiffness considerations,explained in section 5.2; and the pre-contact brakingforce, explained in section 5.3.5.1 A 6-DOF Spring-Damper SystemConceptually, a copy of the haptic handle is placed in the virtualscene and is coupled to the dynamic object through a springdamperconnection, as shown in Figure 7.The real haptic handle controls the position and orientation ofits virtual counterpart. This influences the spring’s displacement,which generates a virtual force/torque on the dynamic object andan opposite force/torque on the real haptic handle. Spring displacementalso includes rotational motion, as shown in Figure 7 bythe spiral at the center of the dynamic object (suggestive of a clockmainspring). Spring force is proportional to displacement, whilespring torque is proportional to the angle of rotation from anequivalent-angle analysis and directed along an equivalent axis ofrotation [11].404A45

Haptic HandleWhenever many point-voxel intersections occur simultaneously,the net stiffness may become so large as to provoke haptic instabilitiesassociated with fixed-time-step numerical integration. Todcope with this problem, we replace the vector sum of all pointvoxelforces by their average, i.e., divide the total force by the cur-k Tb rent number of point-voxel intersections, N. This introduces forceTdiscontinuities as N varies with time, especially for small values of−F springN, which degrades haptic stability. We mitigate this side effect bydeferring the averaging process until N = 10 is reached:k ifRb F springF Net = F Total N < 10RmFF Net = -------------- Totalif N ≥ 10N ⁄ 10Dynamic Objectand similarly for torque. K ff is adjusted to assure reasonably stablenumerical integration for the fixed time step and at least 10Figure 7. Dynamic model based on virtual coupling.simultaneous point-voxel intersections. While this heuristic leadsThis 6-DOF spring makes the dynamic object tend to acquireto relatively satisfactory results, we are investigating a hybrid ofthe same position and orientation of the virtual haptic handle,constraint-based and penalty-based approaches that formallyassuming that the two objects are initially registered in some manner,e.g., with the center of the handle located at the dynamicaddress both the high-stiffness problem and its dual of low stiffnessbut high mechanical advantage. Forcing an object into a narrowwedge-shaped cavity is an example of the latter problem.object’s center of mass and the handle’s main axis aligned withone of the dynamic object’s principal axes. The virtual object isDynamic simulation is subject to the well studied problem ofassigned mass properties, which are reflected at the haptic interfaceas apparent mass that is added to the haptic device’s intrinsicnon-passivity, which might be defined as the unintended generationof excessive virtual energy [2,10]. In a haptic system, nonpassivitymanifests itself as distracting forces and motions (nota-inertia. We operated at a small reflected mass of 12 g. The forceand torque equations used here are:bly, vibrations) with no apparent basis in the virtual scenario. Nonpassivityis inherent in the use of time-sampled penalty forces andF spring = k T d – b T vin the force discontinuity that is likely to occur whenever a pointτ spring = k R θ – b R ωcrosses a voxel boundary. Another potential source of non-passivityis insufficient physical damping in the haptic device [10]. Evenwherek T , b T = spring translational stiffness and viscositya relatively passive dynamic simulation may become highly nonpassivewhen placed in closed-loop interaction with a haptick R , b R = spring rotational stiffness and viscositydevice, depending on various details of the haptic device’s design,θ = equivalent-axis angle (including axis direction)its current kinematic posture, and even the user’s motion behavior.The most direct way to control non-passivity is to operate at thev , ω = dynamic object’s relative linear and angular velocity. highest possible force-torque update rate supported by the hapticSpring stiffness is set to a reasonably high value that is stilldevice, which for our work was the relatively high value of 1000comfortably consistent with stable numerical behavior at theHz. We also investigated the technique of computationally detectingand dissipating excessive virtual energy. While this had someknown time sampling rate. Stiffness and viscosity are straightforwardlyrelated to obtain critically damped behavior. A limitationsuccess, it was eventually replaced by the simpler technique ofof this simple formalism is that it is only valid for a dynamicempirically determining the largest value of K ff consistent withobject having equal moments of inertia in every direction, such asstable operation over the entire workspace of the haptic device. Asa sphere of uniform mass density. Since we were not interested ina further refinement, we discovered some residual instability in thereflected moments of inertia, and indeed sought to minimize them,dynamic object when it lies in free space. Whenever that occurs,this was an acceptable limitation. It represents an implicit constrainton the virtual object’s mass density distribution but not ontherefore, we apply zero force and torque to the haptic device(overriding any non-zero spring values). A free-space configurationis trivially detected as every point of the dynamic object inter-its geometrical shape.secting a free-space voxel of the environment.5.2 Virtual Stiffness Considerations5.3 Pre-Contact Braking ForceWhen the virtual object is in resting contact with the half-voxeldeepforce field described by stiffness K , we want to prevent the The treatment of spring-force clamping in section 5.2 ignoredffuser from stretching the spring so far as to overcome the force field the fact that the dynamic object’s momentum may induce deeperand drag the dynamic object through it. The spring force is instantaneous point-voxel penetration than is possible under restingcontact, thereby overcoming the force field. Currently, we doclamped to its value at a displacement of s/2, where s is the voxelsize. In the worst case, this contact force is entirely due to a single not attempt to avoid this outcome in every instance. Instead, wepoint-voxel interaction, which therefore determines an upper limit generate a force in the proximity voxel layer that acts to reduce theon the spring force. This can be viewed as a modification of the point’s velocity, called the pre-contact braking force. In order togod-object concept [25], in which the god-object is allowed to avoid a surface stickiness effect, the force must only act when thepenetrate a surface by up to a half voxel instead of being analyticallyconstrained to that surface.determine whether the point is approaching or receding, consultpoint is approaching contact, not receding from a prior contact. Toits405A46

associated inward-pointing surface normal, nˆ i , and then calculatethe force:F i = – bv i (–nˆ i ⋅ vˆi ), if nˆ i ⋅ vˆi < 0F i = 0, if nˆ i ⋅ vˆi ≥ 0where b is a “braking viscosity,” v i is the velocity of the i th pointin the point shell, and vˆi is a unit vector along v i .As a simple heuristic, therefore, adjust b so as to dissipate theobject’s translational kinetic energy along the direction ofapproaching contact within one haptic cycle:1(--mv 2 ) ⁄ ∆t2b = -------------------------------------v ⋅ Σ i v i (– nˆ i ⋅ vˆi )where m and v are the dynamic object’s mass and velocity componentalong ΣF i , respectively, and the sum over i is understood totraverse only points for which nˆ i ⋅ vˆi < 0 .We have not yet implemented a braking torque. Calculating thistype of torque would be similar in form to the translational brakingviscosity equation above.A weakness of the braking technique is that an individualpoint’s velocity may become so large that the point skips over theproximity voxel in a single haptic cycle, or even worse, over allvoxels of a thin object. We call this the “tunnelling problem.” Thisis particularly likely to happen for points of a long dynamic objectthat is rotated with sufficient angular velocity. One possible solutionis to constrain the dynamic object’s translational and angularvelocities such that no point’s velocity ever exceeds s ⁄ ∆t .6. RESULTSThe system configuration for our preliminary implementation isillustrated in Figure 8. Haptic rendering is performed on a dedicatedhaptics processor, which asserts updated force and torqueinformation to the haptic device and reads position and orientationof the haptic handle in a closed loop running at 1000 Hz.SGIGraphicsGraphics: 20−60Hz update ratePCGraphicsControllerFigure 8. System Configuration.6−DOFPHANTOMHaptics: 1000Hz update rateIn a separate asynchronous open loop, the haptics processortransmits UDP packets containing position and orientation informationto a dedicated graphics processor, which renders theupdated scene at about 20 Hz. This section provides more detailson the system components and presents some preliminary results.6.1 Haptics DeviceWe used a desk-mounted system called the PHANTOM Premium6-DOF Prototype (shown in Figure 9), made by SensAbleTechnologies, Inc. This system includes the mechanism, its powerelectronics, a PCI interface card, and the GHOST ® SoftwareDeveloper’s Kit (SDK). Force feedback in three translationaldegrees-of-freedom is provided by a vertical 2-link planar structure,with a third orthogonal rotational axis at the base. Cabletransmission actuators drive linkages from the base. Its peak forceis 22 N and the nominal positioning resolution is 0.025 mm at theend effector. The translational range of motion is about 42×59×82cm, approximating the natural range of motion of the entire humanarm. Torque feedback in three rotational degrees of freedom isprovided by a powered gimbal mechanism that provides torques inyaw, pitch, and roll directions. Its peak torque is 0.67 Nm and thenominal resolution is 0.013˚ in each axis. The rotational range ofmotion is 330˚ in both yaw and roll, and 220˚ in pitch.Figure 9. User with the 6-DOF haptic device.Low-level interactions with the PCI interface card are handledby the PHANTOM device drivers provided with the system. TheGHOST SDK transparently provides real-time motion control,including the use of a proprietary mechanism that guarantees a 1kHz servo rate. A kinematic model that deals with conversionsbetween joint space and Cartesian space, and dynamics algorithmsthat optimize the feel by compensating for device dynamics.Although the GHOST SDK supports numerous high-level interactionswith the system, our usage is currently limited to (1) queryingfor global position and orientation of the end effector as a 4×4homogeneous transformation matrix and (2) asserting the desiredglobal force and torque.6.2 Haptics and Graphics ProcessingThe dedicated haptics processor of our prototype system was a350 MHz Pentium ® II CPU with 128 MB of RAM running WindowsNT ® . The functions of voxelization, voxel-sampling, andforce generation were provided by Boeing developed softwareknown as Voxmap PointShell, which implements the approachpresented in this paper. Voxmap PointShell is interfaced with theGHOST SDK, which manages the 1 kHz servo loop. Within thisloop, the haptic handle’s position and velocity information isreceived, a haptic frame is rendered, and updated force and torqueinformation is sent to the device. GHOST monitors the time consumptionof each loop and interrupts operation whenever a 1 kHzservo loop constraint is violated. Outside the servo loop, a separate,asynchronous loop samples the transformation matrices forthe dynamic object and haptic handle, and sends them via UDP toa dedicated graphics processor.Our dedicated graphics processor was an SGI Octane withone 250 MHz R10000 processor, 256 MB of RAM, and SI graphics.For visualization we use FlyThru ® , a proprietary high-performancevisualization system. This system was first used to virtually406A47

“preassemble” the Boeing 777 and is now employed on commercial,military, and space programs throughout Boeing. FlyThru canmaintain a frame rate of ~20 Hz, independent of the amount ofstatic geometry. This is achieved by rendering the static geometryonce to the color and Z-buffers, then reusing those images for subsequentframes [20]. This visualization scheme provided smoothmotion with no noticeable lag.One disadvantage of using two separate computers is that setupand usage tend to be cumbersome. In light of this, we have alsoimplemented our approach on an Octane with two processors —one used strictly for haptics and the other for graphics.6.3 Virtual ScenarioThe static environment of our virtual scenario consisted of simulatedaircraft geometry, with beams, tubes, wires, etc., voxelizedat 5 mm resolution. Its polyhedral representation contains 593,409polygons. Its FlyThru representation consumed 26 MB of memory,and its voxelized representation consumed 21 MB. Voxelizationtime on a 250 MHz SGI Octane was 70 sec. A closeup shot ofa dynamic object (a teapot) maneuvering through a portion of thisenvironment is shown in Figure 10.The dynamic object for much of our testing was a small teapot(75 mm from spout to handle), logically representing a small toolor part, which when voxelized at 5 mm resolution yielded 380points in its pointshell for the PC haptics processor. The dedicatedhaptics processor of the two-processor Octane system was able toachieve a maximum of 600 points for the same object.Figure 10. Dynamic object in the test environment.6.4 Preliminary Test ResultsWe haptically rendered the motion of the teapot through thesimulated aircraft geometry, paying particular attention to motionbehavior and quality of force feedback. We evaluated the feelingof free space as well as resting and sliding contact (with the forcefield). In an attempt to explore the system’s limits, we sought toinduce haptic instabilities and exact-surface interpenetrations bytrapping the teapot in congested areas and by staging high-speedcollisions.Subjectively, the observed free-space behavior was indistinguishablefrom power-off operation, for translational as well asrotational motion. Sliding behavior on a flat or slowly curving surfacewas notably smooth. A relatively slight surface roughness wasfelt when sliding in contact with two surfaces. Torques wereclearly felt. We were able to move the teapot easily through congestedareas where combinations of rotation and translation wererequired to find a path through the area, similar to path planningfor maintenance access.Throughout such investigation, a 1 kHz update requirement wasmaintained. We were unable to cause the teapot to pass completelythrough any of the environment surfaces, including relatively thinones, even at maximum collision speed. There were remarkablyfew potential exact-surface interpenetration events. One naturalmetric is the ratio of penetration to collision events (PR) defined asthe number of haptic frames registering one or more potentialexact-surface penetrations divided by the number of haptic framesregistering contact with the force-field layer (including penetrationevents).We evaluated the benefit of the pre-contact braking force byselectively disabling it and re-measuring PR. The effect of this wasfewer exact-surface penetrations, as shown in Table 2.Table 2. Penetration ratioTest Braking Penetrations Contacts PR1 No 70 69,0002 Yes 6 108,0001.0 × 10 – 36×10 – 5All such work was done with the haptic device limited to 15 Nforce and 0.1 Nm torque. At these limits we found the device to bestable for every possible type of motion.7. CONCLUSIONS AND FUTURE WORKThe voxel-based approach to haptic rendering presented hereenables 6-DOF manipulation of a modestly sized rigid objectwithin an arbitrarily complex environment of static objects. Thesize of the moving object (i.e., the number of points in the pointshell) is limited by the processor speed, while the size of the staticenvironment is limited by memory. A force model was describedin which the interaction of the moving object’s surface normalswith the static voxmap was used to create haptic forces andtorques. Results of testing an implementation of our approach on a6-DOF haptic device showed that the performance appears to beacceptable for maintenance and assembly task simulations, providedthat the task can tolerate voxel level accuracy.It is apparent to us that we are just beginning to discover all thepotential uses for the voxmap sampling method in haptics andother fields. Our primary focus will be to enhance the performanceof the system for use in complex environments.The voxel sampling method can be easily parallelized, usingclones of the static environment and cyclic decomposition of thedynamic object’s pointshell. We intend to take advantage of this byinvestigating parallel computing environments, specifically lowlatencycluster computing. This will allow haptic simulation oflarger and more complex dynamic objects.Another area of interest that we are pursuing involves usingwider-bit-width voxel types (4-bit, 8-bit, etc.). This enhancementwill allow for an extended force field range to model compliancewhen simulating varying material types.We also intend to continue investigating solutions to problematicsituations, like the wedge problem and tunnelling (movingthrough a thin object without detecting collision), as well as furtherreducing non-passivity.407A48

AcknowledgmentsThe authors express their thanks to colleagues Karel Zikan forthe idea of voxel sampling, Jeff A. Heisserman for the idea of normal-alignedforce direction, Robert A. Perry for creating simulatedaircraft geometry, and Elaine Chen of SensAble Technologies, Inc.for literature research and technical information about the PHAN-TOM device and GHOST software support.References[1] Adachi, T., Kumano, T., Ogino, K., “Intermediate Representationsfor Stiff Virtual Objects,” Proc. IEEE Virtual RealityAnnual Intl. Symposium, pp. 203-210, 1995.[2] Adams, R.J. and Hannaford, B., “A Two-Port Framework forthe Design of Unconditionally Stable Haptic Interfaces,” Proc.IROS, Anaheim CA, 1998.[3] Avila, R.S. and Sobierajski, L.M., “A Haptic InteractionMethod for Volume Visualization,” Proc. Visualization’96, pp.197-204, Oct. 1996.[4] Baraff, D., “Curved Surfaces and Coherence for Non-PenetratingRigid Body Simulation,” Computer Graphics (proc.SIGGRAPH 90), vol 24, no. 4, pp. 19-28, Aug. 1990.[5] Baraff, D., “Fast Contact Force Computation for NonpenetratingRigid Bodies,” Computer Graphics (proc. SIGGRAPH94), pp. 23-42, July 1994.[6] Berkelman, P.J. and Hollis, R.L., “Dynamic performance of ahemispherical magnetic levitation haptic interface device,” inSPIE Int. Symposium on Intelligent Systems and IntelligentManufacturing, (Proc. SPIE), Vol. 3602, Greensburg PA,Sept. 1997.[7] Brooks, F.P., Ouh-Young, M., Batter, J.J., Jerome, P., “ProjectGROPE — Haptic Displays for Scientific Visualization,”Computer Graphics (proc. SIGGRAPH 90), pp. 177-185,Aug. 1990.[8] Cohen, J.D., Lin, M.C., Manocha, D., and Ponamgi, M.K., “I-COLLIDE: An Interactive and Exact Collision Detection Systemfor Large-Scale Environments,” Computer Graphics(proc. SIGGRAPH 95), pp. 189-196, Aug. 1995.[9] Clover, C.L., Control system design for robots used in simulatingdynamic force and moment interaction in virtual realityapplications, Ph.D. thesis, Iowa State University, Ames, IA,Apr. 1996.[10]Colgate, J.E., Grafing, P.E., Stanley, M.C., and Schenkel, G.,“Implementation of Stiff Virtual Walls in Force-ReflectingInterfaces,” Proc. IEEE Virtual Reality Annual InternationalSymposium (VRAIS), Seattle, WA, pp. 202-208, Sept., 1993.[11]Craig, J.J., Introduction to Robotics: Mechanics and Control.2nd ed., Addison-Wesley, Reading MA, 1989.[12]Garcia-Alonso, A., Serrano, N., and Flaquer J., “Solving theCollision Detection Problem,” IEEE Computer Graphics andApplications, vol. 14, no. 3, pp. 36-43, 1994.[13]Gottschalk, S., Lin, M.C., Manocha, D., “OBBTree: A HierarchicalStructure for Rapid Interference Detection,” ComputerGraphics (proc. SIGGRAPH 96), pp. 171-180, Aug. 1996.[14]Jackins, C., and Tanimoto, S.L., “Oct-Trees and Their Use inRepresenting Three-Dimensional Objects,” Computer Graphicsand Image Processing, vol. 14, no. 3, pp. 249-270, 1980.[15]Kaufman, A., Cohen, D., Yagle, R., “Volume Graphics,” IEEEComputer, 26(7), pp. 51-64, July, 1993.[16]Logan, I.P., Wills D.P.M., Avis N.J., Mohsen, A.M.M.A., andSherman, K.P., “Virtual Environment Knee ArthroscopyTraining System,” Society for Computer Simulation, SimulationSeries, vol. 28, no. 4, pp. 17-22, 1996.[17]Mark, W.R., Randolph, S.C., Finch, M., Van Verth, J.M., andTaylor II, R.M., “Adding Force Feedback to Graphics Systems:Issues and Solutions,” Computer Graphics (proc. SIG-GRAPH 96), pp. 447-452, Aug. 1996.[18]Massie, T.H. and Salisbury, J.K., “The Phantom Haptic Interface:A Device for Probing Virtual Objects,” Proc. of theASME International Mechanical Engineering Congress andExhibition, Chicago, pp. 295-302, 1994.[19]Mirtich, B. and Canny, J., “Impulse-based Dynamic Simulation.”Proceedings of Workshop on Algorithmic Foundationsof Robotics, Feb. 1994.[20]OpenGL Architecture Review Board, Woo, M., Neider, J., andDavis, T. OpenGL Programming Guide, 2nd, Addison-Wesley,Reading, MA, 1997.[21]Ruspini, D.C., Kolarov, K., and Khatib, O., “The Haptic Displayof Complex Graphical Environments,” Computer Graphics(Proc. SIGGRAPH 97), pp. 345-352, Aug. 1997.[22]Sclaroff, S. and Pentland, A., “Generalized Implicit Functionsfor Computer Graphics,” Computer Graphics (Proc. SIG-GRAPH 96), pp. 247-250, July, 1991.[23]Witkin A. and Welch, W., “Fast Animation and Control ofNonrigid Structures,” Computer Graphics (Proc. SIGGRAPH90), pp. 243-252, Aug. 1990.[24]Yokokohji, Y., Hollis, R.L., and Kanade, T., “What you cansee is what you can feel. Development of a visual/haptic interfaceto virtual environment,” Proc. IEEE Virtual RealityAnnual Int. Symposium (VRAIS), pp. 46-53, Mar., 1996.[25]Zilles, C.B. and Salisbury, J.K., “A Constraint-based GodobjectMethod for Haptics Display,” Proc. IEEE/RSJ Int.Conf. on Intelligent Robots and Systems, Pittsburgh, PA, pp.146-151, 1995.408A49

Advances in Voxel-Based 6-DOF Haptic RenderingWilliam A. McNeely, Kevin D. Puterbaugh, James J. TroyBoeing Phantom WorksApril 13, 2005AbstractAn approach is presented for realizing an order-of-magnitude improvement in spatial accuracy forvoxel-based 6-DOF haptics. It trades constant-time performance for greater spatial accuracy. Thishelps to make 6-DOF haptics applicable to extraordinarily complex real-world task simulations, whichoften admit no other known solution short of physical mockup. A reduction of haptic fidelity is tacticallyincurred but simultaneously mitigated by augmenting standard voxel-sampling methodology withdistance fields, temporal coherence, and culling of redundant polyhedral surface interactions. This isapplied to large-scale haptic scenarios involving multiple moving objects and to collaborative virtualenvironments.Keywords: Haptics, physically based modeling, collision detection, voxel sampling, collaborativevirtual environments1. IntroductionThe voxel sampling approach to 6-DOF haptics demonstrated the potential for simulating real-worldtasks such as maintainability assessment of engineering design [13]. In its simplest form, voxel samplingalso provides constant-time performance, which directly solves the problem of providing reliable1000Hz haptic refresh rates without requiring decoupled simulation and haptic loops, which in turnavoids intermediate representations [14]. However, constant-time performance exacts a steep price inspatial accuracy. While some real-world tasks can be simulated adequately using, say, 10~15mm voxelsfor a scenario with the size and complexity of an automobile engine compartment, many more taskswould become accessible at smaller voxel size such as 1mm or even less. Since the number of surfacepoint samples in the moving objects varies inversely as the square of voxel size, scenarios withrealistically sized moving objects and/or small voxel size may easily exceed a million points. However,modern processors can perform only about 2000 point-voxel intersection tests per haptic frame, whichfalls short of satisfying constant-time performance by two orders of magnitude. Spatial accuracy is soimportant to real-world applicability that alternatives to constant-time performance should be sought.We found that the spatial accuracy of voxel sampling may be improved by an order of magnitude, stillwithout requiring decoupling the simulation and haptic loops, by sacrificing constant-time performancewhile introducing a suite of performance-enhancing measures. The most conspicuous cost of thisA50

approach is a scenario-dependent viscous-like feeling at the haptic interface, although this is made moreacceptable by an enhancement to haptic stability. The performance-enhancing measures presented hereinclude:Distance fields (discussed in Section 3)"Geometrical awareness," which culls point-voxel samples by reducing surface-contactredundance that is inherited from the underlying polyhedral representations (Section 4)Temporal coherence based on distance fields, dead reckoning, and the voxel tree that underlies thesurface point samples of the moving object (Section 5)The best measure of success of this approach is that it enables the haptic simulation of exceedinglycomplex tasks that cannot be simulated by any other known means short of physical mockup. This hasbeen demonstrated for a series of real-world engineering tasks in aerospace and automotive applications.A part removal task from one of these environments will be discussed in this paper.This complex tradeoff is proving acceptable in design-oriented applications such as assessingmaintainability and producibility for complex mechanical assemblies, and indeed, it is enabling a newlevel of functional capability.In addition to performance-enhancing measures, the paper also discusses some of our findingsassociated with implementing advanced voxel-based haptics for single and multi-user applications(Section 6) and results of performance testing for complex virtual environments (Section 7).2. Related WorkModeling with polygons offers greater spatial accuracy than can be attained using voxels. However,polygon-based 6-DOF haptics is subject to severe performance barriers that limit its practicalapplicability, for example, by imposing a convexity requirement that constrains scenarios to 10~100 pairsof convex primitives [8]. A single concave object such as a dish antenna or a helical-spiral tube maydecompose into hundreds or thousands of convex primitives, depending on modeling accuracy.Polygonal decimation may be used to reduce the number of convex primitives, but at the cost ofcompromising polygonal accuracy.NURBS-based modeling offer superior spatial accuracy for 6-DOF haptics, but at the cost of evengreater performance barriers and shape constraints. This approach has so far demonstrated the ability tosimulate only relatively low-complexity scenarios that may not contain surface slope discontinuitiessuch as sharp edges [16].The voxel sampling approach of [13] imposes no formal shape constraints nor complexity constraints onthe static objects. In this paper we adopt major elements of that approach, including 3-level 512-treevoxel hierarchy and tangent-plane force model. However, we abandon the goal of constant-timeperformance, because it limits scenarios to poor spatial accuracy and/or small moving objects, and wecompensate by introducing a suite of performance-enhancing measures. The main cost of this change isto incur a tradeoff between the number of points in the moving object and haptic fidelity, but this isacceptable for our purposes, because it enables greater spatial accuracy and/or larger moving objects.The approach of [13] may be extended straightforwardly to support multiple moving objects [20] as wellA51

as multi-user collaborative virtual environments, which will be discussed later in this paper. Quasi-staticapproximation has been introduced into voxel sampling [21], which avoids the need for acontact-stiffness-limiting heuristic, although it sacrifices dynamic realism. In other voxel-based hapticsimplementations, [19] presents a method for improving the pointshell accuracy and device stability, and[18] discusses a method to reduce the number of collision tests for multiple object pairs.Geometrical awareness was exploited as core of the state-of-art feature-based collision detectionalgorithms between polyhedral objects [5][6][12][15]. Geometrical awareness is most often used tocompute the distance between two disjoint polyhedra by considering only the two nearest features(vertex, edge, or face) between them. This lends itself to temporal coherence by tracking only the mostrecent nearest feature pairs. Most implementations impose a convexity requirement, but this is notstrictly necessary, and so we avoid it. Another significant difference is our use of voxel-based distancefields to avoid expensive on-the-fly computation of feature-separation distance.Like voxel sampling, the sensation preserving method of [17] achieves 6-DOF haptic update rates forcomplex objects by trading accuracy for speed while maintaining similar interaction forces. This methodpre-processes the objects by breaking them down into convex pieces and multiple levels-of-detail(LOD). One of the compromises is that its filtered edge collapse technique allows for someobject-to-object interpenetration. Pair-wise testing for multi-object collision is also used in this method.In another 6-DOF method, [10] developed a technique to compute local minimum distances (LMDs)using spatialized normal cone hierarchies. Gradient search techniques have been used to refine theprocess to achieve haptic rates for moderately sized moving objects. The technique requires apre-processing step to produce the spatial hierarchy.Unlike our approach in which voxels replace polygons as representation primitives, voxels may be usedto accelerate the search for intersecting polygons in a polygon-based approach [3]. However, anypolygon-based approach incurs the performance barrier mentioned earlier, e.g., simulation update rateslimited to low hundreds of Hz and moving objects limited to low tens of thousands of polygons. Thetactics of loop decoupling and intermediate representations are helpful in this context [14], but they alsodegrade motion realism and incur the risk of force artifacts.In the area of collaborative virtual environments, [4] developed a time-delayed collaborative hapticsapplication where one user at a time had control of a single moving object. A shared virtual environmentwith simultaneous haptic interaction was demonstrated by [11] over long distances. Stable simulationwas achieved with 150-200 ms of one way delay, but was limited to point contact interaction. Aroom-sized simulation trainer with 6-DOF force feedback was built by [9] for astronaut EVA thatallowed two crew members to cooperatively manipulate the same large object. Simultaneous interactionwith time delay was not taken into account.2.1 Voxmap PointShell ReviewThis section is intended to help clarify and review some of the concepts of voxel-based collisiondetection associated with our earlier Voxmap PointShell TM (VPS) paper [13].Figure 1 shows the three basic steps associated with our voxel-based haptics process. The first step isdetermining which points of an object’s pointshell representation have come into contact with thevoxelized representation of another object (a). The magnitude and direction of the collision forces areA52

then determined. This process uses a model in which a force vector is calculated based on the depth ofpenetration along a normal perpendicular to a plane tangent to the object’s surface at each point (b). Thesum of the collision forces are applied to the dynamic object and numerically integrated. The resultingmotion is transferred though a virtual coupling (c) to generate the forces presented to the haptic deviceand the user. (A multi-user version of this technique will be discussed later in this paper.)VPS works equally well for convex and concave objects, and it does not require decoupling the hapticand simulation loops. It uses a 3-level voxel hierarchy based on a 512-tree, where the leaf node is voxel,the middle level is called "chunk" and contains 512 voxels, and the highest level is called "hyperchunk"and contains 512 chunks.(a) (b)(c)Figure 1. (a) Point-voxel collision detection, (b) Tangent plane force model, (c) Virtual couplingOne of the aspects of this method that needs some clarification is the issue of multiple moving parts. Theoriginal VPS paper described the interaction between a pair of objects, where one object is representedas a collection of surface points and the other as a group of voxels. A common misconception is that oneof the objects is not allowed to move. This is incorrect -- both objects in the collision pair are allowed tomove.All objects are voxelized in their initial positions in the global frame. Now consider the collision of apair of objects that are both moving. In order to avoid the computationally expensive step ofre-voxelizing objects on the fly, the positions and velocities of the points of the pointshell object (thesmaller of the two, for performance reasons) are transformed into the rest frame of the voxmap object.The collision is analyzed in that frame, and the resulting force and torque are transformed back into theglobal frame and applied to the objects in their current positions.A53

For the simplest case, where only one of the objects is moving, the voxel object’s coordinate systemaligns with the world coordinate system and additional coordinate transformations are not required. Forsimplicity, this was the situation described in the earlier paper, but in the general case where bothobjects may be moving, the transformation to the voxel object’s coordinate system will be needed.Also, either object in the collision pair can be a combination of multiple parts that have been mergedtogether to create a single logical object. Merged objects still maintain individual visual attributes, likecolor, but merged parts behave as a single entity from the collision detection method’s point of view.Environments with more than one pair of moving parts are also possible. The main issue is maintaininga list of collision pairs. For the worst case, the number of object pairs that need to be tested at eachupdate is:Npairs = n(n-1)/2where n is the number of moving objects.In the VPS method, each object is treated dynamically as a free object for the duration of a time step,subject to forces of collision with neighboring objects plus any external forces. This is only anapproximation, of course, but it becomes a better approximation as the time step decreases, and itasymptotically approaches correct multi-body dynamic behavior in the limit of zero time step.Other moving object issues associated with multi-user collaborative virtual environments will bediscussed in Section 6.13. Distance FieldsIt is useful to have advance warning of potential contact between pointshell and voxmap objects. Forexample, such warning is required by the temporal coherence technique described in Section 5. For thatreason we extend the voxelization of an object beyond its surface into free space surrounding the object,marking such free-space voxels with integer values that represent a conservative estimate ofdistance-to-surface expressed in units of voxel size. This creates a voxel based distance field, asillustrated in the 2D example of Figure 2.A54

Figure 2. Voxel based distance field (in 2D)We employ a simple "chess-board" distance-transformation algorithm [2] to calculate the distance field,which gives a conservative estimate of Euclidean distance along non-axis-aligned directions.VPS supports 2, 4, or 8 bit voxels. The smallest positive value(s) are conventionally reserved for interiorvoxels, which in Figure 2 are marked 1. The distance field extends out to a user-specified maximumvalue, constrained only by the integer range.Unless noted otherwise, we assume the use of 4-bit voxels in this paper, since that is a practical choicefor haptic applications in current computing environments. For 4-bit voxels the outermost positivevoxels could be marked with values up to 15, representing a distance-to-surface of 13 voxels. However,the hierarchical extension of temporal coherence (Section 5.1) works optimally when the maximumdistance-to-surface is matched to the power of the voxel hierarchy. Since we use a 512-tree [13], and512 is the cube power of 8, the optimum maximum distance-to-surface is 8, corresponding to voxelsmarked 10 (since surface voxels are marked 2). Consequently, values 11 through 15 of the 4-bit rangeare unused.The "geometrical awareness" technique described in Section 4 requires three different types of distancefield, based on distance to selected geometrical features (vertex, edge, or face). Each field isindependently pre-computed and packed into a word. For 4-bit voxels this implies 16-bit words, wherethe remaining 4 bits are unused. When discussing voxel bitwidth one must be careful to specify whetherit refers to the bitwidth of an individual distance field or, less rigorously, to the size of the word requiredto store all three distance fields. Whenever the expression "16-bit voxels" is used in this paper, it refersto 16-bit words containing three distance fields of 4 bits each.3.1 Other Proximity-Based ApplicationsOne distance field applications that we have developed (and is now part of the VPS API) is a functionthat colors vertices of the dynamic object model based on its proximity to other objects. Figure 3 showsdistance fields used for proximity-based coloring (warmer colors indicate closer proximity). The mainA55

enefit from proximity coloring is that it aids haptic interaction by visually conveying distance tocontact.Figure 3. Voxel-based proximity coloringApplications that use static environment voxel data are also possible. Highlighting surface voxels withina specific distance to the moving object produces a shadow-like effect that can also aid indistance-to-contact perception. Proximity-based distance measurement can be used to give a reasonableapproximation for quickly determining minimum distances. The distance gradient information couldalso be useful for path planning, similar to potential field based path planning applications.3.2 Collision OffsettingForces are generated using the tangent plane model, as reviewed in Section 2.1. One is free to select thevoxel layer in which tangent-plane forces are generated, which we refer to here as the "force layer." Ifone selects surface voxels for that purpose, then as a potentially undesirable side effect, the exact surfaceof the pointshell object (e.g., its polygonal representation) may, in general, interpenetrate the exactsurface of the voxmap object. In order to minimally avoid exact-surface interpenetration, one must adoptthe second layer of free-space voxels as the force layer [13]. Since the distance field extends farther thantwo layers into free space, one may move the force layer to even more distant free-space layers andthereby create a collision offsetting effect. This is useful in task simulations where additional clearanceis needed but is not formally modeled, e.g., to allow for human grasp in a part-manipulation task. InVPS one can dynamically vary the force layer and thereby dynamically vary the amount of clearance.One might consider extending this scheme to the pointshell. The pointshell is normally derived from thecenterpoints of surface voxels, but a free-space voxel layer might also be used for that purpose.However, free-space layers contain more voxels than the surface layer, and VPS performance degradesas pointshell size increases. For that reason, VPS derives the pointshell from the surface layer, except inthe situation that the user requests a range of collision offsetting that exceeds what is achievable byA56

dynamically varying the force layer inside the voxmap object. In that case, VPS derives the pointshellfrom the free-space layer that is both nearest the surface and minimally satisfies the user’s requestedrange of collision offsetting.Despite the static nature of the pointshell as described above, it is possible to dynamically vary thelocations of the points in the pointshell, by displacing them a short distance along the direction of thesurface normal, either toward free space or toward the interior of the object. This provides the capabilityof fine-tuning the amount of collision offsetting. However, this has the problem that, depending on thedirection of displacement and the local curvature of the surface, the displaced points may spread apart,creating a looser mesh of points that runs the risk of undetected penetration. One way to counteract thiseffect is to select a voxel size for the pointshell object that is smaller than that of the voxmap object, atthe price of tactically degrading VPS performance.An interesting application of pointshell displacement is mating-surface simulation, as illustrated inResults (Section 7) for a simple ball-and-socket scenario. In general, mating-surface simulation isproblematic at haptic speeds, in the absence of kinematical constraints or similar special-caseinformation, because manifold surface contact is computationally expensive. If mating parts arepermanently constrained within a mechanism for the entire duration of a motion scenario, thenkinematical constraints are certainly appropriate. However, it becomes problematic when kinematicalconstraints may engage or disengage during a simulation. For example, if a wrench can be used on acertain type of fastener, then the simulating system must know that association in advance. Anysubsequent changes to tool or part geometry are liable to invalidate that association. Furthermore, thesimulating system must somehow decide when to engage the constraint and when to disengage it, e.g.,by detecting that the tool is sufficiently aligned with the part to engage the kinematical constraint. Thisleads to artifacts such as a mysterious attractive force that acts to seat the tool whenever is it sufficientlyaligned with the part. Another artifact is a sticky feeling when trying to disengage the tool. VPS suggestsan approach, albeit a computationally expensive one, to avoid such problems and artifacts by avoidingkinematic constraints altogether 1 .4. Geometrical AwarenessAlthough the approach presented here is voxel-based, voxels may inherit properties of their parentpolyhedral objects at discretization time, which has great value in culling point-voxel intersections at runtime, as explained below.To begin, consider the interaction of a pair of rigid non-penetrating polyhedral objects. Consider theirsurfaces as a pair of point manifolds that exhibit an arbitrary (even infinite) number of pointintersections (surface contacts) for a given configuration. For physically-based modeling purposes, theonly interesting contacts are those where one or both points belong to a C1 discontinuity in theirrespective parent surface. As a simple 2D example, the only interesting contacts between two blocks aretheir vertex-edge contacts, as illustrated in Figure 4.A57

Figure 4. One 2D block rests upon another 2D block. (The red circles represent vertex-edge contacts.)In 3D, only vertex-surface and edge-edge contacts are interesting. ("Surface" is understood to include itsedge boundaries and "edge" its vertex boundaries, hence edge-vertex and vertex-vertex contacts are bothtrivial subsets of edge-edge.) We refer to this powerful insight as geometrical awareness, to adopt theterminology of [5]. This result is entirely general for non-penetrating polyhedral objects, in particular, itdoes not require convexity. One may ignore all surface-surface and surface-edge contacts, whicheffectively reduces the problem’s dimensionality and reduces computational load enormously.Geometrical awareness can be applied to voxel sampling as follows. Point samples are taken as thecenter points of surface voxels. One labels each point as vertex, edge, or surface, according to whetherits parent voxel inherited as a "priority feature" the vertex, edge, or surface attribute, respectively, fromthe underlying polyhedral geometry. By "priority feature" we mean the following priority ordering offeature inheritance. If a point’s parent voxel intersects (i.e., contains) one or more vertices in thepolyhedral geometry, then the point is labeled as vertex, even if its voxel also intersects edge or surfaceelements. Similarly, an edge point’s voxel intersects one or more edges but no vertex, while a surfacepoint’s voxel intersects one or more surfaces but neither edge nor vertex.To more efficiently apply geometrical awareness to point-voxel interactions such as in the tangent-planeforce model, we precompute three voxel-based distance fields toward the nearest surface-, edge-, andvertex-voxel, respectively, as described below. Thus, one uses surface points to sample thevertex-distance field, vertex points to sample the surface-distance field, and edge points to sample theedge-distance field.A known limitation of geometrical awareness is that it is not effective against manifold contact of 3Dedges (e.g., a sword’s edge perfectly aligned with another sword’s edge). In that case, geometricalawareness prescribes testing a potentially large number of point-voxel contacts along the linear region ofoverlap. It is not clear how to generalize geometrical awareness so as to address both the common formof edge-edge contact (e.g., swords crossed at an angle) and the exotic case of edge-edge congruency.Fortunately, the latter almost never occurs in practical scenarios, not even within the accuracy of a voxelsize.4.1 Optimizing Voxel/Polygonal AccuracyFeature-based distance fields are most effective when the accuracy of the underlying polyhedralgeometry matches voxel accuracy, for the following reason. As one increases polyhedral accuracy(holding voxel size constant), one obtains more polygons of smaller dimensions, which increases thelikelihood that a given voxel will contain a vertex and/or an edge. That increases the number ofvertex-surface and edge-edge interactions at the expense of surface-surface interactions, which tends todefeat geometrical awareness and degrade performance. To compound matters, polyhedral accuracy istypically much better than voxel accuracy. Often it is decided casually, e.g. in the process of exporting itfrom a CAD system, oblivious to voxel size.A58

For best results, therefore, polyhedral accuracy must be reduced to voxel accuracy. We accomplish thisthrough a process similar to decimation, at voxelization time, as follows. First, tessellate the polyhedralfacets into triangles. Then, if any pair of adjacent triangles has the property that its non-shared verticesdeviate from coplanarity by less than 1/2 voxel size, and also if their polyhedral angle is less than 90degrees, then that pair of triangles is treated as a single quasi-planar quadrilateral for voxelizationpurposes. Otherwise, if those criteria are not met, then the pair of triangles remains dissociated. Thisprocess is repeated by considering triangles adjacent to a quasi-planar quadrilateral, which may lead to aquasi-planar pentagon, etc. After all triangles have been so processed, distance fields are constructedfrom the features of the resulting quasi-planar polygons. The 90-degree polyhedral-angle criterionprevents small curved objects (such as a sphere with diameter less than a voxel size) from being reducedto a single planar polygon.5. Temporal CoherenceThe voxel sampling method provides a natural opportunity for exploiting spatial and temporalcoherence, or temporal coherence in short. This is done by tracking and predicting the status of points inthe "pointshell" (set of surface point samples) of the dynamic object. A point that contacted a surfacevoxel in the previous frame is likely to remain in contact in the current frame.Whenever a point samples its appropriate voxel-based distance field (see Section 3), it obtains aconservative estimate of its minimum distance from any contact. If we also know the point’s maximumspeed, then by dead reckoning we can predict how many frames will elapse before contact can possiblyoccur, which allows us to safely reduce the frequency of point sampling. Hence, the pointshell maycontain more points than could possibly all be tested in a single haptic frame, and since the pointshell isderived from surface voxels, this enables the use of smaller voxels and greater spatial accuracy.This requires knowing a point’s maximum speed, but the latter is formally unlimited. A more seriousproblem is that the known speed may be so large that the available processing power cannot keep upwith the burden of predicting contact for all free-space points. To solve these problems, we impose aspeed limit that applies to all points. For this purpose we denote the maximum distance that any pointmay travel in a haptic frame as MaxTravel. In general, MaxTravel is adjusted on a frame-by-frame basisbecause it varies inversely with the number of points that require testing during that frame. As theamount of contact and near-contact increases more point tests become necessary. It is mandatory to testpoints that were in contact in the previous haptic frame. However, free-space points may be scheduledfor testing at a reduced frequency.MaxTravel has an absolute upper bound of 1/2 voxel size 2 , in order to prevent points from skippingover the penalty-force region of surface voxels and penetrating into the object’s interior. Since the timeduration of haptic frames is constant, MaxTravel is equivalent to a speed constraint. This expresses itselfat the haptic interface as a viscous-like resistance whenever the virtual-world speed tries to exceedMaxTravel per haptic frame. For example, consider a scenario modeled using 2mm voxels and 1000Hzhaptic refresh rate. A maximum speed of 1/2 voxel per millisecond is 1 meter/second. This correspondsto user motion of roughly one arm’s length per second, which is unrealistically fast in the context of anyapplication that involves manipulating objects with careful intent. In this simple example, therefore, theMaxTravel constraint has negligible impact at the haptic interface. However, in a more completeA59

analysis (1) the speed constraint applies to every point on the object’s surface, which generates a morecomplicated constraint on the object’s overall translational and rotational velocities, (2) any spatialscaling between the virtual world and the haptic interface must be considered, (3) MaxTravel may besmaller than its absolute upper bound of 1/2 voxel, as calculated below:(1)where nCapacity is the number of point tests that the processor can perform per haptic frame,nMandatory, is the number of "mandatory" tests (for points already in contact), n iis the number ofpoints in free space at i voxels (i>0) from contact, and s is voxel size. If Equation 1 yieldsMaxTravel

Under this implementation, distance-to-contact queues become quite sparse. To accelerate theirtraversal, each queue is ordered into a 2-level hierarchy. The leaf level contains individual bits of thequeue, while the upper level contains bits that are one whenever any of its 32 leaf-level children are one.This enables the skipping of entire 32-bit runs of zero bits. When n iis zero, the entire queue is emptyand may be skipped. While it may not be obvious that this implementation is preferable to moresophisticated point-scheduling schemes that can be imagined, in fact it yielded higher performance thanseveral alternatives that we explored.Temporal coherence conveys an important, if unexpected, benefit for haptic stability. Under virtualcoupling (Figure 1c), the most likely source of instability is large transient movements of the dynamicobject. However, MaxTravel inherently prevents large transient movements. Stability is a very complextopic, and there are many other possible sources of instability (e.g., limit-cycle oscillations, overly stiffvirtual systems, unpredictable user-applied forces, device limitations, etc.). However, empirically, thestability benefit from MaxTravel has enabled perfectly stable haptic operation for all scenarios that wehave ever tested.5.1 Hierarchical Temporal CoherenceSince the pointshell is derived from the centroids of surface voxels, it inherits the spatial hierarchy of itsparent voxel tree. All points that came from the same "chunk" of the voxel tree (the first level above leaflevel) are assigned to contiguous bit addresses in the distance-to-contact queues. Then, whenever theentire chunk’s worth of points is known to lie in free space, we may remove all such points from theirqueues and continue tracking only the chunk’s distance-to-contact, e.g., by testing the chunk’s centroidagainst the surface distance field. (Since the chunk’s contents may be marked with a mixture of surface,edge, and vertex attributes, we must test against the most conservative distance field, which is thesurface distance field.) This greatly reduces the point-testing burden, since in a 512-tree, a chunkcontains about 100 points on average.One may learn whether a chunk’s entire point contents lie in free space as follows. Chunks are markedwith a discretized distance-to-contact value in the same manner as voxels, thereby creating a chunk-leveldistance field. The pointshell-object’s chunk centroid is then used to sample the static-object’schunk-level distance field, in precisely the same manner as point-voxel sampling. If such a test revealsthat a chunk lies beyond the space spanned by voxel-level distance fields, then that chunk is consideredto lie entirely in free space, and chunk-level temporal coherence is applied. On the other hand, if apreviously free-space chunk enters the space spanned by voxel-level distance fields, then its contents aredisgorged and re-inserted into the point queues. (The cost of such transitions may be greatly reduced byexploiting the fact that the points have contiguous bit addresses.)Point sampling and chunk-centroid sampling behave identically in all respects except the following."Contact" is re-defined to mean that the chunk enters the space spanned by voxel-level distance fields, asdescribed above. Every chunk that lies in that space is considered to occupy a mandatory chunk queue.MaxTravel is modified straightforwardly in Equation 1 by augmenting nMandatory with achunk-specific contribution and also extending the summation over queues to include the newchunk-level queues.5.2 Point driftingAs a significant performance optimization, one may reduce the frequency of voxmap lookup duringA61

point testing, as follows. Whenever voxmap lookup becomes necessary (as explained below), the point’scurrent exact spatial position is stored, along with its current voxel-accurate distance-to-contact (asdiscovered through voxmap lookup and expressed implicitly by the point’s distance-to-contact queuenumber). Subsequently, whenever that point falls due for testing under temporal coherence, one firstcomputes its point drift, defined as the exact distance between its current position and its previouslystored position. If so much drift has occurred that the point may be "too near contact" (as definedbelow), then voxmap lookup becomes necessary and drifting begins anew. Otherwise, if the amount ofdrift is not so great, then voxmap lookup is avoided, and the point is allowed to continue drifting. Thecriterion for being "too near contact" is that the point could possibly have drifted as much as two queuesaway from contact. In principle, one could more aggressively wait until it was only one queue fromcontact, but we elect to have a one-queue margin of safety.When a point begins drifting, it stays in its initial distance-to-contact queue until the amount of pointdrift warrants re-queueing, e.g., due to drifting more than a voxel size. Whenever re-queueing becomesnecessary, we conservatively assume that the point moved nearer to contact, i.e., to a lower-numberedqueue. That incrementally increases the frequency of testing, but empirically, each test suddenlybecomes about 7 times faster by avoiding voxmap lookup. This 7-fold advantage decreases as driftingproceeds, becoming minimal when the point drifts as near as two queues from contact, but when thathappens the point is re-tested subject to voxmap lookup and properly re-queued, and drifting beginsanew. The net performance benefit of point drifting depends in a complicated way on the motionscenario, but typically it is several-fold.5.3 Dynamic Pre-Fetching of Voxel DataIt may easily happen that there is insufficient system memory to hold all voxel data for a given scenario,especially for large-scale scenarios and/or small voxel sizes. Under 32-bit operating systems theaddressing limit is 4GB, which is often reduced further to 3GB or 2GB. While virtual memory is a goodsolution for non-time-critical applications, it is fundamentally incompatible with time-critical haptics.Just-in-time memory paging causes highly distracting force discontinuities or even haptic-controllertimeouts. To avoid such adverse effects, one needs a predictive memory-paging scheme. For that reason,we implemented a dual-thread scheme that supports time-critical operation at haptic rates in one thread,coupled with dynamic pre-fetching of voxel data in the other thread.A convenient way to implement dynamic pre-fetching is to integrate it with chunk-level temporalcoherence as described in Section 5.1. The latter includes probing the space that lies beyond the spacespanned by voxel-bearing chunks in the static distance fields. Consequently, one can readily detect whena given chunk of the dynamic object has reached a distance of one chunk size away from anyvoxel-bearing chunk(s) in the static distance fields. Whenever that happens (to recall Section 5.1), oneimmediately switches level-of-detail representations in the dynamic object, from using the chunk’scentroid to using its constituent points. To extend that mechanism to dynamic pre-fetching, simply treatsuch representation-switching events as requests that voxel-bearing chunk(s) of the static distance fieldsshould be fetched into real memory, if necessary. A separate thread can then perform such fetching intime to satisfy access by the haptic thread.There is no way to guarantee that a pre-fetching thread can always act fast enough to satisfy the hapticsthread, depending on the speed of the hard drives, scenario complexity, the backlog of pre-fetchingrequests, size of MaxTravel compared to chunk size, etc. To cover all such contingencies, we allow thehaptics thread to be temporarily suspended as needed to allow the pre-fetching thread to catch up.A62

During a state of suspension, MaxTravel is set to zero, and no forces are sent to the haptic interface. Welimit the duration of any suspension to 2 seconds, after which the simulation is terminated. Empirically,even with our largest scenarios, such suspensions occur so rarely and/or have such brief duration thatthey proved imperceptible. Furthermore, we have not yet encountered a scenario that was prematurelyterminated by the 2-second timeout.We did not extend this mechanism to hyperchunks, nor was temporal coherence extended tohyperchunks, on the grounds that the complexity of such an extension seemed to outweigh its potentialbenefits.6. ImplementationWe have used a variety of architectures for experimentation and prototyping, using either one or twocomputers. For production use we implement the VPS collision detection and force generationalgorithms in a separate computer we call the "Haptic Controller" using a client-server model. Ourchoice for this approach was driven by the mismatch between the computing requirements of physicallybased modeling and the available workstations used by typical engineering departments. Physicallybased modeling has these characteristics:Computationally intensive -- dual CPU’s are best so one can be devoted to haptics and the other tosecondary tasksLarge amounts of memory (RAM) are requiredLarge amounts of available high speed disk space are needed to save voxel dataProduction workstation installations generally have these characteristics:A single CPUModest amounts of memoryComputation is already taxed by graphical renderingMemory fills with data representations optimized for graphical renderingLocal disk space may be lower speed or inaccessibleOS and application software installation tightly controlled by IT departmentThe mismatch between requirements and existing hardware is solved by putting the haptic process on aPC that is devoted to haptic processing. The haptic device (or other type of input device) is thenconnected to this PC as shown in Figure 5. The Haptic Controller PC is connected to the clientworkstation via ethernet and TCP/IP. If the PC is given an IP address on the same subnet as theworkstation, connecting them via a switch minimizes bandwidth contention and allows them tocommunicate at 100Mbit/second, regardless of the network connection speed available to theworkstation (often much slower). The Haptic Controller PC has no visual interaction with the user, andneed not have an associated monitor. The Haptic Controller supports a variety of interaction devicesincluding: various models of the PHANTOM haptic device, 6-DOF Spaceball (and similar) devices withno force feedback, and a 2-DOF mouse with no force feedback.A63

Figure 5. Haptic Controller configurationWithin the Haptic Controller, one thread is devoted to collision detection and force generation, and asecond thread handles communication tasks with the client, and pre-processing. When the Spaceball isused, a third thread receives updates from it.The Haptic Controller provides these services to the client: voxelization, transparently caches voxel datafor later reuse, manages the haptic device, and supplies updated positions of the goal and moving objectson demand. An API is supplied for use by the host application. The API is designed to minimize theintrusion into the host application.We have used the Haptic Controller with two applications: FlyThru ® , a Boeing-proprietary visualizationsystem used for design reviews, and a prototype application used for investigating collaborative haptics.The results reported here were obtained with FlyThru. FlyThru is designed to handle large amounts ofgeometry, and includes rendering optimization for the special case of a fixed eye point, and a smallamount of moving geometry. This optimization is important because it allows the environment to berendered in full detail during haptic interaction at responsive frame rates.High-speed hard drives are desirable for the Haptic Controller for the sake of dynamically pre-fetchingvoxel data (Section 5.3). Empirically, hard drives with higher data transfer rates (like 10k-15k RPMSCSI drives) are more likely to meet pre-fetching demands for large-scale scenarios. If lower-speed harddrives are used, then haptic force quality acquires a rough and viscous feeling whenever two objectsmake contact for the first time, due to the fact that MaxTravel is set to zero while waiting for voxel datato appear in memory.6.1 VPS-Based Collaborative Virtual EnvironmentsIn addition to building VPS-based applications with multiple constrained and unconstrained movingobjects, we have recently implemented a multi-user environment for collaborative 6-DOF haptics thatuses VPS for collision detection and response. The types of haptically enabled collaboration applicationsthat we have been investigating include: design reviews, maintenance access, and training.Implementing a collaborative virtual environment (CVE) with multiple simultaneous haptic usersbecomes more difficult when users are located at geographically separate sites. Haptic interaction is verysensitive to synchronization delays produced by communication over large distances. In order tomaintain haptic stability, while minimizing the impact on interactive performance, the application needsto be designed with time delay compensation in mind. In our CVE implementation, we address the delayissue by using peer-to-peer communication and a multi-user virtual coupling configuration. Figure 6A64

shows our collaborative virtual environment application for maintenance access analysis.Figure 6. Haptic enabled collaborative virtual environmentThe peer-to-peer architecture synchronizes the CVE without a central server 3 . Each user is running aseparate simulation of the environment in which models and motions are synchronized with the otherusers. The implementation uses TCP packets between the front-end graphical interface and UDP packetsbetween haptic controllers. The system supports active users with haptics and non-haptic devices, aswell as passive (visual only) users. A user can enter and leave the simulation at any time withoutimpacting the other users.The two main types of collaborative tasks that we have focused on are those involving: (1) each usercontrolling separate objects, and (2) multiple users controlling the same object. We will refer to these astype-1 and type-2, respectively. Both have the same type of infrastructure with respect to data and modelsynchronization, network connections, and device control. There are some significant differences aswell.The first type (control of different objects) has the same pair-wise collision checking requirementsdiscussed in Section 2.1, but with the added requirement that users be aware that a voxel size mismatchbetween users will produce an asymmetric force response. A user with a smaller voxel size than otherusers will create an imbalance in contact forces between objects. This allows user A’s pointshell objectto contact user B’s voxmap and generate repulsive forces before B’s pointshell object makes contactwith A’s voxmap. This gives the user with the smaller voxels an enhanced ability to push/pull otherusers around without being affected equally by their interactions. Although the exact nature of thisimbalance is probably unique to voxel-based haptics, this type of condition is a common problem incollaborative systems without centralized management -- for example, in a multi-player video gameusers can cheat by modifying the local front-end interface to give themselves special powers. In general,collaborative haptics applications will have asymmetric behavior if force calculation parameters are notthe same for all users.The second type of collaboration (users controlling the same object) requires a new type of couplingconnection. In previous implementations [13], we have used virtual coupling elements [1][7] to connectthe dynamic object to the haptic device. For the multi-user case, the virtual coupling model wasextended to connect the instances of the object that all users control. Since each user is running anA65

independent simulation, there is an instance of the object independently calculated for each simulation.Coupling effects from the other instances of the object act as additional external forces on the localdynamic simulation of each object instance. Figure 7 shows this connection for a two user arrangement.Figure 7. Multi-user connection model using virtual coupling elementsThe multi-user virtual coupling effectively creates an environment for bilateral teleoperation of multiplehaptic (or robotic) devices, with the addition of collision detection and response from objects andconstraints in a virtual environment. One of the interaction drawbacks of this method is the potential fordivergence of the multiple object instances. This can occur when another object (like a thin wall) getstrapped between the instances of the dynamic object.Another interesting finding for both of these approaches to collaboration is that the haptic devices anddynamics simulations remain stable when force information from the other users is transmitted at ratesbelow 1000Hz. The systems were functionally stable when external force updates from the other userswere received at 100Hz. Note, we still maintained each users local simulation at 1000Hz to keepnumerical integration and haptic loops stable.A combined environment that simultaneously allows both types of interaction presents some interestingresponse possibilities. For example, what happens when two users are controlling one object (type-2)and then a third user joins the environment and controls another object (type -1)? In addition to feelingbilateral forces from each other, the first two users will see and feel contact interaction with the third asexpected with type-1 contact. From the third user’s point of view, he or she will see and interact withwhat appears to be a single instance of a moving object -- unless the users controlling that object enterinto a divergent condition. One option for dealing with this situation is to allow user 3 to see and interactwith both instances of that object. How well this works from a usability standpoint is still unknown,since a combined environment is not something we have tested yet.In addition to multi-user interaction issues, time delay compensation is another major concern incollaborative virtual environments. Time delay is especially problematic when users are located atgeographically separate sites. It is less critical for the type-1 collaboration, where the non-coupled usersmay not be aware of the delay -- at least not initially. They will still see the other users objects movingand instantly feel forces when they make contact with those objects. The delay will become apparentwhen objects have continuous contact. Although local contact forces are felt immediately, the reactionof the other user’s object to the contact is delayed. A similar delayed reaction occurs when the contact isA66

emoved. Fortunately, this delay does not appear to destabilize the simulations. But that is not the casefor type-2 collaboration.When multiple users simultaneously control the same object, time delay can cause the haptic devices tobecome unstable. For this situation, we have implemented a method for linking the current value of thetime delay to the stiffness gains in the cross-user virtual coupling. A linear reduction of the stiffness fordelays up to 1 second appears to keep both simulations stable. At this point, we are using gains that havebeen determined experimentally. We hope to develop a more theoretical basics for gain selection in thefuture.6.2 Physically Based Modeling Without Force FeedbackAlthough we have found that task performance of force feedback applications is superior tophysically-based applications without force feedback, the cost of haptic devices appears to be a barrierto widespread adoption in the engineering community. Devices that have 6-DOF input, but no forcefeedback, like the Spaceball ® provide a low cost alternative. However, just as with a haptic device, theVPS algorithms prevent part interpenetration and impose physically possible motion. The Spaceball is aforce input device where the user pushes/pulls against internal spring-like elements. This motion is thenconverted into position and orientation data. Other 6-DOF, non-force feedback devices like theMicroScribe ® have also been successfully tested with VPS-based virtual environments.So far, our informal testing results indicate that these types of devices are adequate for tasks of low tomoderate difficulty. We have found that very difficult tasks, like part extraction from congestedenvironments, are only solvable with force feedback. In general, task performance is usually slowerwithout force feedback, but is a reasonable alternative for some conditions. In our implementations, weusually attach the Spaceball to the Haptic Controller PC rather than the host workstation so that anyexisting use of another Spaceball by the host application (i.e., for view control) is unaffected. With sucha system, an ambidextrous user can simultaneously change the viewpoint and manipulate objects.7. Experimental ResultsOur high-performance haptic rendering system has been implemented on Linux ® , Microsoft Windows ® ,and SGI IRIX ® . The performance results in the following discussion were obtained using a twoprocessor 2.8 GHz Xeon PC with 2GB of RAM running Windows XP. Haptic rendering is performed onone processor to provide updated force and torque information to the haptic device and read position andorientation of the haptic handle in a closed-loop control system running at 1000Hz. Force feedback isprovided by a PHANTOM ® Premium 1.5/6-DOF haptic interface made by SensAble Technologies, Inc.The host graphics application for these experiments is FlyThru ® , our internal high-performancevisualization system, which uses a second computer to maintain a graphics frame rate of 10~20 Hz,depending on the complexity of the moving geometry, but independent of the amount of static geometry.This high performance is achieved by rendering the static geometry once to the color and Z-buffers, thenreusing those images for subsequent frames [22]. (Other graphics display environments are alsopossible.)VPS provides the capability to crudely simulate mating-surface scenarios without using kinematicconstraints, as described in Section 3.2. This is illustrated here for the simple scenario of a ball that mayA67

e rotated in a cradle-like socket (Figure 8a). This example illustrates a worse case scenario where alarge amount of object-to-object contact occurs. In this case, the ball is the pointshell object, and itspoints are displaced by half a voxel toward the interior of the ball, in order to allow the ball to seat fullywith the socket. For this scenario we measure VPS performance in terms of the time required for a fullrotation of the ball. With a radius of 25 mm and a voxel size of 0.35 mm, this takes 1.28 seconds on a2.8GHz processor. The speed of rotation is limited by MaxTravel, which is determined by voxel size andprocessor speed. In this scenario there are, on average, 250 points in contact at all times.Figure 8b shows the 777 Main Landing Gear used here as an example of a large dataset for maintenanceanalysis tasks. The overall dimensions of the this dataset are approximately 4.1 x 1.9 x 4.8 m. Thedynamic object chosen for testing is a large hydraulic actuator near the bottom of the scene thatmeasures 0.9 x 0.2 x 0.2 m. For this test scenario, the user interacts with the environment by removingthe dynamic object from its installed position. Simulation accuracy was adjusted over multiple tests byvarying the voxel size.(a)(b)Figure 8. Models used for testing: (a) Ball and socket model, (b) 777 Main landing gear (with dynamicobject)Table 1 collects the parameters of the dynamic object and the static environments in each of the abovetwo scenarios, in which our approach was able to maintain a 1000Hz haptic refresh rate. Each scenariowas evaluated twice, once with a relatively large voxel size and once with a small voxel size in relationto the overall dimensions of the scene. The Table includes the sampling resolution (voxel size), numbersof triangles, number of sampling points in each dynamic object, numbers of triangles, and number ofvoxels in each static environment.Table 1. Virtual Scenario Measurements 4A68

One cannot straightforwardly assess the relative performance benefits of geometrical awareness andtemporal coherence, since they depend sensitively on the motion scenario. However, one may directlycompare the currently attainable accuracy (as represented by voxel size) against what was attainablebefore the advent of algorithmic enhancements such as geometrical awareness and temporal coherence.The maximum number of points that VPS could process in 1999 was reported as 600 [13]. Currentlythere is no formal limit, but up to 1M points is readily attainable and usable. We must also account forthe fact that CPU speeds have increased about 8-fold since 1999. Consequently, 1M points wasequivalent to 125,000 points in 1999, a 200-fold increase. Since the number of points varies inversely asthe square of voxel size, a 200-fold increase in pointshell capacity corresponds to a 14-fold improvementin accuracy due to VPS algorithmic enhancements alone. Combining this with the CPU-speed increase,there has been a net 40-fold improvement in accuracy since 1999.Throughout testing, we paid particular attention to motion behavior and quality of force and torquefeedback. Artificial viscosity caused by MaxTravel (Section 5) was evident, especially at smaller voxelsizes, whenever objects were in contact or nearly so. However, both force and torque feedback aredistinctly helpful to performing task simulations.These results are from experiments performed in a single user environment, but the performance shouldbe nearly identical in the multi-user environment described above, since each user will be running anidentical simulation (with a small amount of communications related overhead).8. Summary and ConclusionsIn this paper we discussed geometric awareness, temporal coherence, and dynamic pre-fetchingtechniques for improving speed and accuracy of the Voxmap PointShell collision detection method for6-DOF haptic rendering. The desired order-of-magnitude improvement in spatial accuracy was realized,at a cost of reduced haptic fidelity that is proving acceptable. Results from performance tests conductedon large datasets were presented.Additional VPS capabilities for distance fields and surface offsetting were discussed that can be used toenhance collision detection and response, and also provide capabilities for path planning and otherproximity related applications.The use of VPS for multiple moving objects and shared objects in a collaborative virtual environmentwas discussed, as were collaboration related issues unique to voxel-based haptics.A69

We believe that haptic feedback will always remain a powerful adjunct to visual feedback, especially inbusy environments with much occlusion.Footnotes:1. Developers are still free to create additional constraints on top of the basic VPS collision detectionimplementation.2. If the objects are sufficiently far apart, this upper bound may increase to ½ hyperchunk size, as aconsequence of Hierarchical Temporal Coherence (Section 5.1)3. The collaborative architecture is peer-to-peer, which should not be confused with the HapticController architecture, which uses a client server model.4. The total number of voxels shown in Table 1 include internal, surface, and distance field voxels(as described in Section 3). When multiple voxel sizes are listed for a scenario, the smaller numberis the point spacing of the dynamic object and the larger number is the voxel size of the staticenvironment.References[1] Adams, R. and Hannaford, B. "A Two-Port Framework for the Design of Unconditionally StableHaptic Interfaces." Proc. IROS, Anaheim CA, 1998.[2] Borgefors, G., "Distance Transformations on Digital Images", Computer Vision Graphics ImageProcessing, v34, pp. 344-371, 1986.[3] Borro, D., Garcia-Alonso, A., and Matey, L., "Approximation of Optimal Voxel Size for CollisionDetection in Maintainability Simulations within Massive Virtual Environments" Computer GraphicsForum, 23(1), p.13, Mar 2004.[4] Buttolo, P., Oboe, R., Hannaford, B., and McNeely W., "Force Feedback in Shared VirtualSimulations." Proc MICAD, Paris, 1996.[5] Choi, M. and Cremer, J., "Geometrically-Aware Interactive Object Manipulation", ComputerGraphics Forum, 19(1), pp. 65-76, 2000.[6] Cohen, J., Lin, M., Manocha, D., and Ponamgi, M., "I-COLLIDE: An Interactive and ExactCollision Detection System for large-Scale Environments", 1995 Symposium on Interactive 3DGraphics, pp. 189-196, 1995.[7] Colgate, J.E., Grafing, P.E., Stanley, M.C., and Schenkel, G., "Implementation of Stiff Virtual Wallsin Force-Reflecting Interfaces." PRoc. IEEE Virtual Reality Annual Internation Symposium (VRAIS),Seattle WA, pp. 202-208, Sept 1993.[8] Gregory, A., Mascarenhas, A., Ehmann, S., Lin, M., and Manocha, D., "Six Degree-of-FreedomA70

Haptic Display of Polygonal Models", Proc. IEEE Visualization, pp. 139-146, 2000.[9] Homan, D. "Virtual Reality Applications Development", NASA JSC Annual Report for FY-1995 ofthe Automation, Robotics & Simulation Division,http://tommy.jsc.nasa.gov/ARSD/report_FY95/homan_fy955.html, including private correspondence.[10] Johnson, D and Willemsen, P., "Accelerated Haptic Rendering of Polygonal Models through LocalDecent", Int. Symposium on Haptic Interfaces for VR and Teleop Systems, pp. 18-23, Mar 2004.[11] Kim, J., Kim, H., Tay, B.K., Muniyandi, M., Jordan, J., Mortensen, J., Oliveira, M., Slater, M., andSrinivasan, M.A., "Transatlantic Touch: A Study of Haptic Collaboration over Long Distance."Presence, 13(3), pp. 328-337, June 2004.[12] Lin, M., "Efficient Collision Detection for Animation and Robotics", Ph.D. Thesis, University ofCalifornia, Berkeley, 1993.[13] McNeely, W., Puterbaugh, K., and Troy, J., "Six Degree-of-Freedom Haptic Rendering UsingVoxel Sampling", Proc. ACM SIGGRAPH, pp. 401-408, Aug 1999.[14] Mark, W., Randolph, S., Finch, M., Jan Verth, J., and Taylor II, R., "Adding Force Feedback toGraphics Systems: Issues and Solutions", Proc. ACM SIGGRAPH, pp. 447-452, Aug 1996.[15] Mirtich, B., "V-Clip: Fast and Robust Polyhedral Collision Detection", ACM Transactions onGraphics, 17(3), pp. 177-208, 1998.[16] Nelson, D. and Cohen, E., "Optimization-Based Virtual Surface Contact Manipulation at ForceControl Rates", Proc. IEEE Virtual Reality, pp. 37-44, 2000.[17] Otaduy M. and Lin, M., "Sensation Preserving Simplification for Haptic Rendering", Proc. ACMSIGGRAPH, pp. 543-553, Aug 2003.[18] Prior, A. and Haines, K., "The use of a Proximity Agent in a Collaborative Virtual Environmentwith 6 Degrees-of-Freedom Voxel-based Haptic Rendering." Proc. World Haptics Conf., Pisa, Italy, pp.631-632, Mar 2005.[19] Renz, M., Preusche, C., Pötke, M., Kriegel, H.-P., and Hirzinger, G., "Stable Haptic Interactionwith Virtual Environments using an Adapted Voxmap-Pointshell Algorithm" Proc. Eurohaptics,Birmingham, UK, 2001.[20] Troy, J., "Haptic Control of a Simplified Human Model with Multibody Dynamics" Phantom UsersGroup Conf., Aspen, CO, pp. 43-46, Oct 2000.[21] Wan, M. and McNeely, W.A. "Quasi-Static Approximation for 6-DOF Haptic Rendering". Proc.IEEE Visualization conference, Seattle, WA, pp. 257-262, Oct 2003.[22] Woo, M., Neider, J., Davis, T., and Shreiner, D., OpenGL Programming Guide, Version 1.2, 3rd,Addison Wesley, Reading, MA, 1999.A71

Sensation Preserving Simplification for Haptic RenderingMiguel A. Otaduy Ming C. LinDepartment of Computer ScienceUniversity of North Carolina at Chapel Hillhttp://gamma.cs.unc.edu/LODHaptics/AbstractWe introduce a novel “sensation preserving” simplification algorithmfor faster collision queries between two polyhedral objectsin haptic rendering. Given a polyhedral model, we construct a multiresolutionhierarchy using “filtered edge collapse”, subject to constraintsimposed by collision detection. The resulting hierarchy isthen used to compute fast contact response for haptic display. Thecomputation model is inspired by human tactual perception of contactinformation. We have successfully applied and demonstratedthe algorithm on a time-critical collision query framework for hapticallydisplaying complex object-object interaction. Compared toexisting exact contact query algorithms, we observe noticeable performanceimprovement in update rates with little degradation in thehaptic perception of contacts.CR Categories: I.3.5 [Computer Graphics]: ComputationalGeometry and Object Modeling—Hierarchy and GeometricTransformations; I.3.6 [Computer Graphics]: Methodology andTechniques—Interaction Techniques I.3.7 [Computer Graphics]:Three-Dimensional Graphics and Realism—Virtual RealityKeywords: Level-of-Detail Algorithms, Haptics, Collision Detection1 IntroductionHaptic rendering, or force display, is emerging as an alternativeform or an augmentation for information presentation, in additionto visual and auditory rendering. The sense of touch is one of themost important sensory channels, yet it is relatively poorly understoodas a form of human-machine interface. Coupled with graphicalrendering, force feedback can enhance the user’s ability to interactintuitively with complex synthetic environments and increasethe sense of presence in exploring virtual worlds [Brooks, Jr. et al.1990; Mark et al. 1996; Hollerbach et al. 1997; Salisbury 1999].The first step in displaying force and torque between two 3Dvirtual objects is collision query and contact handling. Collisiondetection has been well studied, and many practical techniques andtheoretical advances have been developed (see surveys by Lin andGottschalk [1998] and Klosowski et al. [1998]). Yet, despite thehuge body of literature in this area, the existing algorithms cannotrun at the desired force update rates (at least hundreds of Hz butpreferably several kHz) for haptic rendering of complex models.Figure 1: Adaptive Resolution Selection. Top: Moving jaws incontact, rendered at their highest resolution; Bottom: The appropriateresolution (shown in blue and green) is selected adaptivelyfor each contact location, while the finest resolution is displayed inwireframe.This is mainly due to the fact that the optimal running time of anycollision detection algorithm intrinsically depends on both the inputand output sizes of the problem. Those in turn depend on boththe combinatorial complexity and the contact configuration of theobjects involved in the queries. While we can render millions ofpolygons at interactive rates, we can barely create a force displayof an environment consisting of just tens of thousands of polygonsat the desired update rates.Inspired by the large body of research in digital geometry processingand mesh simplification, we propose an algorithm basedon multiresolution hierarchies of object geometry to perform timecriticalcollision queries for haptic rendering. In addition, ourmethod is influenced by findings from tactual perception and spatialrecognition to preserve pertinent contact information for hapticdisplay.Main Contribution: We introduce the notion of sensation preservingsimplification to accelerate collision queries between twoA72

complex 3D polyhedral models in haptic rendering. Given a polyhedralrepresentation of an object, we generate a series of approximationsfor the object using filtered edge collapse operations, whichsmooth away high-frequency detail in low-resolution approximationswhile respecting the convexity constraints imposed by collisionqueries. Our goal is to generate a multiresolution hierarchy thatcan also be used as a bounding volume hierarchy for time-criticalcontact force computation in haptic rendering. Our computationmodel is based on a criterion that preserves perceivable contact details,effectively making the simplified model feel practically thesame as the original. The resulting multiresolution hierarchy enablesuse of varying resolution approximations in contact queriesat different locations across the surfaces of the objects in contact,depending on each contact configuration, as shown in Fig. 1. Thekey results in this paper include:• A novel simplification algorithm, based on a formal definitionof resolution, to generate representations for contact queries;• A collision detection framework that dynamically selectsadaptive levels of detail at each contact location;• Application of this framework to real-time haptic rendering ofcomplex object-object interaction.This approach allows us to bound both the input and output sizesof the problem, thus achieving the desired contact query performancefor force display. We have applied our approach to hapticrendering of complex models in contact configurations that are particularlychallenging to collision detection algorithms. Comparedto existing exact contact query algorithms, we are able to achieveup to two orders of magnitude performance improvement with littledegradation in the haptic perception of contacts.Organization: The rest of the paper is organized as follows. InSection 2, we give a brief survey of related work. Section 3 presentsthe haptic perception characteristics central to the design of ourcomputational model and the overview of our approach. We describethe construction of the multiresolution hierarchy in Section4 and sensation preserving contact queries using the hierarchy inSection 5. We address implementation issues and present resultsin Section 6. We conclude with a discussion and analysis of ourapproach and implementation, as well as future research directions.2 Previous WorkOur research draws on a large collection of knowledge in mesh simplification,signal processing for digital geometry, collision detection,and haptic display. We briefly survey related work here.2.1 Polygonal SimplificationPolygonal simplification has been an active research topic for thelast decade. Numerous approaches have been proposed. We referreaders to an excellent new book on this subject [Luebke et al.2002]. However, note that the growing interest in perception-basedsimplification for interactive rendering, e.g. [Luebke and Hallen2001], has been based on human visual perceptual metrics. Our approachdiffers in many ways from existing work in this area, and ourtarget application, haptic rendering, has a much higher performancerequirement than visual display. Although we precompute the levelof detail (LOD) hierarchy offline, the way we select the appropriateLOD on the fly is “contact-dependent” at each contact locationacross the object surfaces. Our approach bears a closer resemblanceto view-dependent simplification [Luebke et al. 2002], which useshigher resolution representations on the silhouette of the object andmuch coarser approximations on the rest of the object that is not asnoticeable to the viewpoint.2.2 Signal Processing for Digital GeometryMuch of the work presented in this paper takes advantage of observationsmade in signal processing of meshes, since many conceptsin multiresolution representations can be analyzed using frequencydomain analysis. By generalizing discrete Fourier analysisto meshes, Taubin [1995] introduced a novel linear-time low-passfiltering algorithm for surface smoothing. This algorithm can beextended to accommodate different types of geometric constraintsas well. Through a non-uniform relaxation procedure, whoseweights depend on the geometry instead of connectivity, Guskovet al. [1999] generalized signal processing tools to irregular trianglemeshes. Our work borrows some ideas from the relaxationtechniques proposed in this paper.2.3 Collision DetectionHierarchical data structures have been widely used to design efficientalgorithms for interference detection between geometric models(see surveys by Lin and Gottschalk [1998] and Klosowski etal. [1998]). Typical examples of bounding volumes include axisalignedboxes and spheres, chosen for their simplicity in performingoverlap tests between two such volumes. Other hierarchies includek-d trees and octrees, OBBTree, cone-trees, R-trees and theirvariants, trees based on S-bounds, etc. [Lin and Gottschalk 1998;Klosowski et al. 1998]. Additional spatial representations are basedon BSP’s and their extensions to multi-space partitions, space-timebounds or four-dimensional tests (see a brief survey by Redon et al.[2002]), and many more.Hubbard [1994] first introduced the concept of time-critical collisiondetection using sphere-trees. Collision queries can be performedas far down the sphere-trees as time allows, without traversingthe entire hierarchy. This concept can be applied to any type ofbounding volume hierarchy (BVH). However, no tight error boundshave been provided using this approach. An error metric is often desirablefor interactive applications to formally and rigorously quantifythe amount of error introduced. Approaches that exploit motioncoherence and hierarchical representations for fast distance computationbetween convex polytopes have been proposed [Guibas et al.1999; Ehmann and Lin 2000]. However, these techniques are onlyapplicable to convex polyhedra.O’Sullivan and Dingliana [2001] studied LOD techniques forcollision simulations and investigated different factors affectingcollision perception, including eccentricity, separation, distractors,causality, and accuracy of simulation results. Based on a model ofhuman visual perception validated by psychophysical experiments,the feasibility of using these factors for scheduling interruptible collisiondetection among large numbers of visually homogeneous objectsis demonstrated. Instead of addressing the scheduling of multiplecollision events among many objects, we focus on the problemof contact queries between two highly complex objects. Our approach,guided by a completely different tactual perception for hapticrendering, has distinct goals differing significantly from theirs.Recently, GPU accelerated techniques have also been proposedfor collision queries [Lombardo et al. 1999; Hoff et al. 2001].Though fast, these approaches are not currently suitable for hapticrendering, since the readback from framebuffer and depth buffercannot be done fast enough to perform queries at haptic updaterates.2.4 HapticsOver the last few years, haptic rendering of geometric models hasreceived much attention. Most previous work addresses issues relatedto rendering the interaction between a probe point and 3Dobjects [Ruspini et al. 1997]. This problem is characterized byhigh spatial coherence, and its computation can be localized. ByA73

contrast, we attack the problem of force rendering for arbitrary 3Dpolyhedral object-object interaction, which involves a substantiallyhigher computational complexity. Force rendering of object-objectinteraction also makes it much more challenging to correctly cacheresults from previous computations.McNeely et al. [1999] proposed “point-voxel sampling”, a discretizedapproximation technique for contact queries that generatespoints on moving objects and voxels on static geometry. This approximationalgorithm is the first to offer run-time performance independentof the environment’s input size by sampling the objectgeometry at a resolution that the given processor can handle. A recentapproach proposed by Gregory et al. [2000] is limited to hapticdisplay of object-object interaction for relatively simple modelsthat can be easily represented as unions of convex pieces. Kim et al.[2002] attempt to increase the stability of the force feedback usingcontact clustering, but their algorithm for contact queries suffersfrom the same computational complexity.The idea of using multiresolution representations for haptic renderinghas been recently investigated by several researchers. Paiand Reissel [1997] investigated the use of multiresolution imagecurves for 2D haptic interaction. El-Sana and Varsheny [2000] proposedthe construction of a multiresolution hierarchy of the modelduring preprocessing. At run-time, a high-detail representation isused for regions around the probe pointer and a coarser representationfarther away. The proposed approach only applies to hapticrendering using a point probe exploring a 3D model. It does not extendnaturally to force display of two interacting 3D objects, sincemultiple disjoint contacts can occur simultaneously at widely varyinglocations without much spatial coherence. The latter problem isthe focus of our paper.3 OverviewIn this section, we first present important findings from studies ontactual perception that guide our computational model. Then, wedescribe the requirements for haptic rendering and our design goals.3.1 Haptic Perception of Surface DetailFrom a perceptual perspective, both formal studies and experimentalobservations have been made regarding the impact of contactareas and relative size (or curvature) of features to the size of thecontact probe (or finger) on identifying fine surface features.Klatzky and Lederman [1995] conducted and documented studieson identification of objects using “haptic glance”, a brief hapticexposure that placed several temporal and spatial constraints onstimulus processing. They showed that a larger contact surface areahelped in the identification of textures or patterns, though it was betterto have a stimulus of the size comparable or just slightly smallerthan that of the contact area when exploring geometric surface features.Okamura and Cutkosky [1999] defined a fine (geometric) surfacefeature based on the ratio of its curvature to the radius of thefingertip acquiring the surface data. Their paper gives examples onhow a larger fingertip, and thus a larger surface contact area, canmiss some surface detail.In this paper, we mainly focus on geometric surface features, notmicroscopic surface roughness or friction. We draw the followingkey observation from these studies relevant to our computationalmodel:Human haptic perception of the existence of geometricsurface features depends on the ratio between the contactarea and the size of the feature, not the absolute sizeof the feature itself.Here we broadly define the size of a given feature in all three dimensions,namely width, length, and height. The width and lengthof a feature can be intuitively considered as the “inverse of resolution”(formally defined in Sec. 4) of the simplified model. Thatis, higher resolution around a local area implies that the width andlength of the geometric surface features in that neighborhood aresmaller, and vice versa. We extend the concept of “height” to includea perceivable amount of surface deviation introduced in thesimplification process, according to haptic perception.Figure 2: Contact area and resolution: (a) high resolution modelwith large contact area; (b) low resolution model with large contactarea; (c) high resolution model with small contact area.As illustrated in Fig. 2, the observation drawn by Okamura andCutkosky [1999] for tactile feedback can extend to haptic renderingof contact forces between rigid bodies. The resolution at which themodels are represented affects the number of contact points usedto describe object interaction. However, increasing the resolutionbeyond a sufficiently large value does not affect the computed netforce much, as shown in Fig. 2(a) and (b).Our proposed model of acceptable error metrics differs notablyfrom that of human visual perception in both the current mesh simplificationliterature and visual collision perception. In visual rendering,a combination of surface deviation (or Hausdorff distance)and the viewing distance from the object is used to determine ifthe representation of the objects requires higher resolution. In hapticrendering, on the other hand, this is governed by the relationshipamong the surface deviation, the resolution of the simplified model,and the contact surface area. We will later show how this relationshiplays the foundation of our algorithmic design and contact queryprocess for haptic rendering in Sec. 4 and Sec. 5.3.2 Requirements and Design DesiderataWe aim to create multiresolution representations where geometricsurface detail is filtered when it cannot be perceived by the sense oftouch. The resulting multiresolution hierarchies can be used to performtime-critical contact queries that stop when the reported resultis accurate up to some tolerance value. This helps to automaticallyspeed up the contact query computation for haptic rendering.In our haptic rendering framework, we have chosen BVHs ofconvex hulls, because overlap tests between convex hulls can beexecuted rapidly in expected constant time with motion coherence[Guibas et al. 1999]. Furthermore, convex hulls provide at leastequally good, if not superior, fitting to the underlying geometry asOBBs [Gottschalk et al. 1996] or k-dops [Klosowski et al. 1998].We integrate BVHs of convex hulls with multiresolution representationsso that the hierarchies, while being used for effective collisiondetection, can themselves be used to report contact pointsand normals with bounded errors at different levels of resolution.To summarize, our goal is to design multiresolution hierarchiesthat:1. Minimize perceptible surface deviation. We achieve thisgoal by filtering the detail at appropriate resolutions and byusing a novel sensation preserving refinement test for collisiondetection;A74

2. Reduce the polygonal complexity of low resolution representations.This objective is achieved by incorporating meshdecimation during the creation of the hierarchy;3. Are themselves BVHs of convex hulls. We perform a surfaceconvex decomposition on the given triangular mesh andmaintain it across the hierarchy. The convex surface decompositionplaces both local and global convexity constraints onthe mesh decimation process.Our algorithm assumes that the input models can be representedas oriented 2-manifold triangular meshes with boundaries.3.3 Notation and TerminologyWe use bold-face letters to distinguish a vector (e.g. a point, normal,etc.) from a scalar value. In Table 1, we enumerate the notationswe use throughout the paper.NotationMeaningr,r i ,r jDifferent resolutionsM k An LOD of a mesh M with a resolution r kc iA convex surface patchC iA convex piece constructed as theconvex hull of a patch c ie(v 1 ,v 2 ) An edge between two vertices v 1 and v 2s,s a ,s bSurface deviationsD,D a ,D bContact areasq A distance query between two convex piecesQ A contact query between two objects thatconsists of multiple distance queries qTable 1: Notation Table4 Multiresolution HierarchyIn this section we describe the hierarchical representations of triangulatedmodels used to perform sensation preserving contactqueries for haptic rendering.We create a hierarchy of static levels of detail (LOD), each levelrepresenting an approximation to the original triangular mesh at adifferent resolution (i.e. spatial scale), to be formally defined next.Because our goal is to provide an error bound arising from contactqueries using simplified models, we must design a multiresolutionhierarchy that computes error metrics between each LOD and theoriginal model.Conceptually, an LOD at resolution r j of a mesh M, M j , canbe obtained from an LOD at a lower resolution r i , M i , by addingdetail at resolutions in the range [r i ,r j ]. Our approach for generatingLODs reverses this definition, so LODs at low resolution areobtained by removing detail at high resolution. While the detail isbeing removed, we quantify it and compute the surface deviation.Following the LOD generation, we obtain a hierarchy where anLOD at resolution r j preserves the lower resolution geometric information,while the higher resolution detail might have been culledaway.We generate each LOD by a sequence of filtered edgecollapse operations (to be defined in Sec. 4.2) that performfiltering and mesh decimation,subject to both local and global convexity constraintsimposed by the collision detection module of the hapticrendering framework.4.1 Definition and Computation of ResolutionBefore we explain how we generate each LOD, we must first formallydefine what we consider as a resolution in our hierarchicalrepresentation. We follow the framework of signal processing forirregular meshes. Our definition of resolution for irregular meshesassumes that a triangular mesh M can be considered as a sampledversion of a smooth surface S, which has been later reconstructedvia linear interpolation. The vertices of the mesh are samples of theoriginal surface, while edges and faces are the result of the reconstruction.Our formal definition of sampling resolution for irregular meshesis based on the 1D setting. For a 1D function F(x), the samplingresolution r is the inverse of the distance between two subsequentsamples on the real line. This distance can also be interpreted asthe projection of the segment between two samples v 1 and v 2 ofthe function on the average value. The average value is the lowresolution representation of the function itself, and can be obtainedby lowpass filtering.Extrapolating this idea to irregular meshes, the sampling resolutionof an edge (v 1 ,v 2 ) of the mesh M at resolution r j , M j , can beestimated as the inverse of the projected length of the edge onto alow resolution representation of the mesh, M i .We locally compute the low resolution mesh M i by filtering themesh M j , applying the filtered edge collapse operation to the edge(v 1 ,v 2 ). Then we compute the normal N of the resulting vertex ṽ 3by averaging the normals of incident triangles. Finally, we projectthe edge on the tangent plane Π defined by N. The resolution r iscomputed as the inverse of the length of the projected edge.1r =(1)‖(v 1 − v 2 ) −((v 1 − v 2 ) · N) · N‖4.2 Filtered Edge CollapseAs stated, our multiresolution hierarchy is obtained through meshsimplification. We have selected edge collapse as the atomic decimationoperation for two main reasons:1. Under the required self-intersection tests, edge collapse canguarantee preservation of topology, a requirement for maintaininga surface convex decomposition of the object duringthe hierarchy construction.2. Topologically, an edge collapse can be regarded as a localdownsampling operation, where two samples (i.e. vertices)are merged into a single one.In the construction of the hierarchy, we aim to:1. Generate multiresolution representations with low polygonalcomplexity at low resolution for accelerating contact queries;2. Filter detail as we compute low resolution LODs. This approachallows more aggressive simplification and enablesfaster merging of convex pieces to build the hierarchy.These two goals are achieved by merging downsampling and filteringoperations in one atomic operation, which we call filterededge collapse.In the filtered edge collapse operation, an edge (v 1 ,v 2 ) is firsttopologically collapsed to a vertex ˆv 3 . This step provides the downsampling.Then, given its connectivity, ˆv 3 is relaxed to a positionṽ 3 , which provides the filtering. In our implementation, we useda relaxation operation based on the minimization of second orderdivided differences [Guskov et al. 1999]. Intuitively, this resemblesthe minimization of dihedral angles, without much affectingthe shape of the triangles. We also tried other filtering techniques,such as those proposed by Taubin [1995], with very similar results.However, linear functions are invariant under the minimization ofsecond order differences. This is consistent with the selection ofA75

the tangent plane of the filtered mesh as the low resolution representationfor the computation of resolutions.In order to apply the relaxation to ˆv 3 , we need to compute a localparameterization. This local parameterization requires an initialposition of ˆv 3 , which is computed using quadric error metrics, proposedby Garland and Heckbert [1997].To sum up, the goal of our simplification and filtering process isto create multiresolution hierarchies for contact queries. As mentionedearlier, the collision detection module imposes convexityconstraints on filtered edge collapse. Next, we will describe howthe convexity constraints are satisfied.4.3 Convexity ConstraintsDue to the requirements of haptic rendering, we have chosen toperform collision detection using the Voronoi marching algorithmand surface convex decomposition as described by Ehmann and Lin[2001], for this approach provides us with both the distance andcontact information needed for force display and its implementationis available to the public. A surface convex decomposition is firstcomputed for the original mesh, and then a hierarchy of convexpieces is created.The surface convex decomposition yields a set of convex surfacepatches {c 1 ,c 2 ,...,c n } [Chazelle et al. 1997; Ehmann and Lin2001]. For the correctness of the collision detection algorithm, theconvex hulls of these patches are computed, resulting in convexpieces {C 1 ,C 2 ,...,C n }.The initial convex decomposition can be created using techniquespresented by Chazelle et al. [1997]. However, our hierarchyis created in a novel way. Instead of creating convex hulls of pairsof convex pieces and joining them into a single convex piece, wemerge neighboring sets of convex patches as long as they representa single valid convex patch. The implications of this procedure forthe collision detection algorithm are explained in Sec. 5.Let c 1 and c 2 be two convex patches of LOD M j . Let c = c 1 ∪c 2be a surface patch of M j . After a filtered edge collapse operationis applied to M j , c 1 and c 2 will be merged if c constitutes a validconvex patch. The convex hull of c, C, becomes the parent of C 1and C 2 in the hierarchy of convex pieces for collision detection.When a filtered edge collapse takes place, the convex patchesin the neighborhood of the collapsed edge may be affected. Theirboundary has to be updated accordingly.A surface convex decomposition for the collision detection algorithmmust meet several constraints:1. All the interior edges of a convex patch must themselves beconvex.2. No vertex in a convex patch may be visible from any face inthe patch, except the ones incident on it.3. The virtual faces added to complete the convex hulls of theconvex patches cannot intersect the mesh.We consider the first constraint as a local constraint and the othertwo as global constraints. Before a filtered edge collapse operationis applied, we must check that the convexity constraints are preservedfor all the convex pieces. Local and global convexity constraintsare treated separately.4.3.1 Local Convexity ConstraintsLet e ≡ (v 1 ,v 2 ) be a candidate edge that will be tested for a filterededge collapse. Let v 3 represent the vertex resulting from theedge collapse, as well as its associated position. The edges in the1-ring neighborhood of v 3 are susceptible to changing from convexto reflex and vice versa. Interior edges of convex patches are convexbefore the filtered edge collapse and must remain convex afterFigure 3: Local Convexity Constraints.it. These constraints can be expressed as linear constraints in theposition of v 3 .Given e, the edge to be collapsed, two possible types of interioredges of convex patches exist: edges incident to v 3 and edges oppositeto v 3 , as shown in Fig. 3. However, both cases can be treatedequally. Assigning v a , v b , v c and v d vertices as in Fig. 3, the convexityconstraint of an edge can be expressed as a negative volumefor the parallelepiped defined by the adjacent triangles:((v b − v a ) ×(v c − v a )) ·(v d − v a ) ≤ 0 (2)To satisfy the convexity constraints, we have opted for formulatingan optimization program, where v 3 is constrained to the segmentbetween ˆv 3 and ṽ 3 , and the objective function is the distanceto ṽ 3 . This optimization program is unidimensional. Because distancein one dimension is linear, it is a simple linear program in onedimension.The position of the result of the constrained filtered edge collapsecan be written as a linear interpolation between the initial positionand the goal position:v 3 = u · ˆv 3 +(1 − u) · ṽ 3 (3)The limit constraints can be expressed as u ≥ 0 and u ≤ 1.The convexity constraints in Eq. 2 can be rewritten as:A · u+B ≥ 0, where (4)A = ((v d − v a ) ×(v c − v a )) ·(ˆv 3 − ṽ 3 )B = ((v d − v a ) ×(v c − v a )) ·(ṽ 3 − v a )v 3 is computed for the minimum value of u that meets all the constraints.When ṽ 3 is not a feasible solution but a solution exists, theconstrained filtered edge collapse can be regarded as a partial filter.4.3.2 Global Convexity ConstraintsThe global convexity constraints are too complicated to be expressedexplicitly, so they cannot be incorporated into the filteringprocess. Instead, they have to be verified after the filtering has beenperformed. We conduct this verification by computing the affectedconvex pieces after the edge collapse and performing the requiredintersection tests, using OBBs [Gottschalk et al. 1996] and spatialpartitioning.If a position v 3 that meets the local convexity constraints hasbeen found, we check the global constraints. If they are met, theedge collapse is valid. If they are not met, then we check them atˆv 3 . If they are not met at ˆv 3 either, the edge collapse is consideredinvalid and we disallow it. If ˆv 3 meets the global constraints, weperform a bisection search between ˆv 3 and v 3 of up to K iterations(in our current implementation K = 3), searching for the positionclosest to ṽ 3 that meets the global convexity constraints, as shownin Fig. 4. v 3 is reassigned to this position.A76

Figure 4: Filtered Edge Collapse with Convexity Constraints.The figure shows a filtered edge collapse where bisection searchis required to find a position that meets the convexity constraints.G and L represent feasible regions of global and local constraintsrespectively.of BVHs, with our simplification processing the different levels ofthe BVH only bound their associated LOD; they do not necessarilybound the original surface. This fact has some implications for thecontact queries, described in Sec. 5.3. The free LODs 11 and 14 inthe figure are obtained through pairwise merging of convex hulls.They serve to complete the BVH, but cannot be considered as LODsof a multiresolution hierarchy. Fig. 6 shows a more detailed view ofthe simplification and merging process. Notice how in the creationof the first LOD, most of the simplification and merging takes placeat the gums. The gums are, indeed, the locations with detail at thehighest resolution. When the processing reaches LOD 7, one toothin particular is covered by a single convex patch, thus showing thesuccess of the processing.4.4 Multiresolution Hierarchy GenerationThe hierarchy of LODs is created by applying successive filterededge collapses on the given mesh, while performing a surface convexdecomposition and merging convex pieces. First we computethe convex decomposition of the initial mesh. We then compute thevalue of resolution for all edges, and set them as valid for collapse.The edges are inserted in a priority queue, where edges with higherresolution have higher priority.The main processing loop always tries to filter and collapse theedge with highest priority. If the filtered edge collapse is successful,the affected edges update their resolution and priority, and they arereset as valid for collapse. Moreover, the filtering and simplificationmay have relaxed some convexity constraints in the neighborhoodof the collapsed edge, so we attempt to merge convex pieces in theprocess as well. If the filtered edge collapse fails, the edge is set asinvalid for collapse. The process continues until no edges are validfor collapse.This process must yield a hierarchy of static LODs. We havedecided to generate a new LOD every time the number of convexpieces is halved. All the pieces in LOD M j that are merged to acommon piece C ∈ M j+1 during the processing will have C as theirparent in the BVH.Ideally, the process will end with one single convex piece, whichserves as the root for the hierarchy to be used in the collision detection.However, this result is rarely achieved in practice, due totopological and geometric constraints that cannot be removed by alocal operation such as filtered edge collapse. In such cases, thehierarchy is completed using a pairwise convex hull merging step.We call these remaining completing LODs “free” LODs.During the process, we assign to each LOD M j an associated resolutionr j . This resolution is the smallest resolution of an edge thathas been collapsed before the LOD M j is generated. Geometricallyit means that the LOD M j preserves all the detail of the originalmesh at a resolution lower than r j . In our sensation preserving simplificationfor haptic rendering, we wish to maximize the resolutionat which LODs are generated. As will be explained in Sec. 5, theperceptual error for haptic rendering is measured by taking into accountthe resolution of the surface detail culled away. By maximizingthe resolution at which LODs are generated, the contact queriescan be completed faster. This is the basis for selecting edge resolutionas the priority for filtered edge collapses. The pseudo code forthe entire process of hierarchy construction is given in Appendix Aon the conference proceedings CD.Fig. 5 shows several of the LODs obtained when processing amodel of a lower jaw (see Sec. 6 for statistics on this model). TheLODs 3 and 6 shown in the figure are obtained from the originalmodel by our simplification process. The convex pieces shown forthe original model are successively merged to create the BVH duringthe process of simplification. Thus, the multiresolution hierarchyitself serves as BVH for collision detection. Unlike other typesFigure 5: Hierarchy of the Lower Jaw. From left to right andtop to bottom, original mesh, LOD 0 , and convex decompositions ofLOD 0 , LOD 3 , LOD 6 , LOD 11 and LOD 14 .Figure 6: Detail View of the Hierarchy. From left to right andtop to bottom, original mesh, LOD 0 , and convex decompositions ofLOD 0 , LOD 1 , LOD 2 , LOD 4 and LOD 7 .4.5 Error MetricsIn this section, we present the parameters that must be computedafter the hierarchy is created, in order to quantify the error for sensationpreserving haptic rendering. The utilization of these parametersduring the contact queries is explained in Sec. 5. To performsensation preserving haptic rendering using a multiresolution hierarchy,we must measure the error that is introduced in the contactquery and force computation and refine the query if the error isabove a given tolerance. Once the hierarchies of LODs are created,with the resolution r computed for each LOD, we must computeseveral additional parameters for measuring the error:A77

1. The surface deviation, s, between every convex patch c andthe original mesh. This is an upper bound on the size of thegeometric surface detail lost during the simplification and filteringprocess.2. A support area, D, for every vertex in the hierarchy. This valueis used to calculate contact area at run-time. The support areaD is computed for every vertex v of the initial mesh M as theprojected area onto the tangent plane of v of the faces incidentto v, such that they are within a distance tolerance from valong the direction of the normal N of v, and their normallies inside the normal cone of N. When an edge (v 1 ,v 2 ) iscollapsed to a vertex v 3 , we assign to v 3 the minimum of thetwo support areas of v 1 and v 2 . We have typically used thesame tolerance used in the contact queries (see Sec. 5) as thedistance tolerance for this computation as well.3. The maximum directed Hausdorff distance, h, computed forevery convex piece C, from the descendant pieces of C.The use of the surface deviation s, the support area D, andthe resolution r of the LODs (whose computation is explained inSec. 4.4) during the contact queries is described in Sec. 5.4. Andthe run-time use of the Hausdorff distance h is described in Sec. 5.3.splitting policy yields a BVTT where the levels of the tree are sortedaccording to their resolution, as shown in Fig. 7. Nodes of theBVTT at coarser resolution are closer to the root. This is a keyissue for optimizing our sensation preserving haptic rendering, becausewe obtain a BVTT where LODs with lower resolution andlarger error are stored closer to the root. Descending the BVTT hasthe effect of selecting finer LODs.As pointed out in Sec. 4.4, the top levels of the BVHs are “free”LODs, which are not obtained using our simplification algorithm,but pairwise convex hull merging. Therefore, the top levels of theBVTT have no associated metric of resolution. The boundary between“free” and regular LODs is indicated in Fig. 8 by the lineλ.5 Contact Computation for HapticsIn this section, we describe how our collision detection algorithmuses the new multiresolution hierarchy described in Sec. 4 to computecontact response for haptic rendering. First, we describe therequirements of our collision detection system. Then, we presentand analyze the data structures and algorithms. Finally, we showhow to perform sensation preserving contact queries for force display.5.1 Basic Haptic Rendering FrameworkOur haptic rendering system uses a penalty-based force computationmodel, in which the amount of force displayed is proportionalto the penetration depth or separation distance. Contact featureswithin a given tolerance value are all considered as “contacts” forthe purpose of force display. For more information about our hapticrendering framework, we refer readers to Appendix B on theconference proceedings CD.We define the contact query between two objects A and B asQ(A,B,δ). From Q(A,B,δ), we obtain all local minima of thedistace function between A and B that are closer than a distancetolerance δ, as well as the associated contact information (i.e. distance,contact normal, etc.). Q(A,B,δ) is performed by recursivelytraversing the bounding volume hierarchies (BVH) of A and B andperforming “distance queries” for pairs of convex pieces. We definethe distance query between two convex pieces a ∈ A and b ∈ B,q(a,b,δ), as a boolean query that returns whether a and b are closerthan δ.5.2 The Bounding Volume Test TreeWe use the concept of the Bounding Volume Test Tree (BVTT)[Larsen et al. 2000] to describe the algorithm and data structuresused in our collision detection system. A node ab in the BVTT encapsulatesa pair of pieces a ∈ A and b ∈ B, which might be testedwith a query q(a,b,δ). Performing a contact query Q(A,B,δ) canbe understood as descending along the BVTT as long as the distancequery q returns “true”. In the actual implementation, theBVTT is constructed dynamically while the contact query is performed.If the distance query result is “true” for a given pair, thenthe piece whose children have the lowest resolution is split. ThisFigure 7: Bounding Volume Test Tree.Instead of starting the contact query Q at the root of the BVTTevery time, temporal coherence can be exploited using “generalizedfront tracking” [Ehmann and Lin 2001]. We store the “front” of theBVTT, F , where the result of the distance query q switches from“true” to “false”, as shown in Fig. 8. The front is recorded at the endof a contact query Q i , and the next query Q i+1 proceeds by startingrecursive distance queries q at every node in the front F .Figure 8: Generalized Front Tracking of the BVTT. The front ofBVTT for the original model, F , is raised up to the new front F ′using mesh simplification, since the contact queries can stop earlierusing the sensation preserving selective refinement criterion. λindicates the portion of the hierarchy constructed using the pairwiseconvex piece merging strategy, instead of mesh simplification,to form bounding volumes in the hierarchy.5.3 Distance Query for Convex PiecesIn a contact query Q using BVHs, we need to ensure that if thedistance query q is “true” for any node of the BVTT, then it mustbe “true” for all its ancestors. To guarantee this result with ourmultiresolution hierarchy, given a distance tolerance δ for a contactquery Q(A,B,δ), the distance tolerance δ ab for a distance queryq(a,b,δ ab ) must be computed as:δ ab = δ + h(a i ,a)+h(b j ,b) (5)A78

where h(a i ,a) and h(b j ,b) are maximum directed Hausdorff distancesfrom the descendant pieces of a and b to a and b respectively.As explained in Sec. 4.5, these Hausdorff distances are precomputed.5.4 Sensation Preserving Selective RefinementThe time spent by a collision query Q depends directly on the numberof nodes visited in the BVTT. Generalized front tracking considerablyreduces the running time of Q when temporal coherenceis high, which is the case in haptic rendering. Then, the time spentby Q is proportional to the size of the front F . However, the costis still O(nm) in the worst case, where n and m are the number ofconvex pieces of the objects.In our system, we further take advantage of the multiresolutionhierarchies to accelerate the performance of the query. The coreidea of our sensation preserving selective refinement is that thenodes of the BVTT are only refined if the missing detail is perceptible.Note that the selective refinement does not apply to the“free” levels of the BVTT. Those levels must always be refined ifthe distance query q returns “true”.As discussed in Sec. 3, the perceptibility of surface features dependson their size and the contact area. We have formalized thisprinciple by devising a heuristic that assigns a functional φ to surfacefeatures which is averaged over the contact area.Given a node ab of the BVTT for which the distance query resultq is “true”, we determine if the missing detail is perceptible bycomputing the functional of the missing detail and averaging it overthe contact area of that node. For a convex piece a of the node ab,with resolution r a and surface deviation from its descendent leavess a , we define the functional φ as:φ a = s ar 2 aThis definition of the functional can be regarded as a measure ofthe maximum volume of features that have been culled away in theconvex piece a.The online computation of the contact area for a pair of convexpieces is too expensive, given the time constraints of haptic rendering.Therefore, we have estimated the contact area by selecting themaximum support areas of the contact primitives (i.e. vertex, edgeor triangle). As explained in Sec. 4.5, the support area D is storedfor all vertices in the hierarchy. For edge or triangle contact primitives,we interpolate the support areas of the end vertices, using thebaricentric coordinates of the contact point.Given functional values of φ a and φ b for the convex pieces aand b, as well as support areas D a and D b , we compute a weightedsurface deviation, s ∗ ab , as:s ∗ ab = max(φ a,φ b )max(D a ,D b )Note that s ∗ ab can be considered as the surface deviation weightedby a constant that depends both on the resolution and the contactarea. If s ∗ ab is above a threshold s 0 , the node ab has to be refined.Otherwise, the missing detail is considered to be imperceptible. Theselection of the value of s 0 is discussed in Sec. 6. As describedin Sec. 4.4 and Sec. 4.5, the resolution r, the surface deviation s,and the support areas D are parameters computed as part of thepreprocessing.By using the described sensation preserving selective refinementof nodes of the BVTT, we achieve varying contact resolutionsacross the surfaces of the interacting objects, as shown in Fig. 1.In other words, every contact is treated independently, and its resolutionis selected to cull away imperceptible local surface detail.As a consequence of the selective refinement, the active front of the(6)(7)BVTT, F ′ , is above the original front F that separates nodes with“true” result for distance queries q from nodes with “false” result.The front does not need to reach the leaves of the BVTT as longas the missed detail is imperceptible, as depicted in Fig. 8. Thisapproach results in a much faster processing of contact queries.5.5 LOD InterpolationA major issue in systems that use multiresolution hierarchies is thediscontinuity that arises when the algorithm switches between differentLODs. This problem is known as “popping” in multiresolution(visual) rendering. In haptic rendering its effects are discontinuitiesin the delivered force and torque, which are perceived by theuser.We have addressed the problems of discontinuities by interpolatingcontact information (e.g. contact normal and distance) fromdifferent LODs. When the sensation preserving selective refinementdetermines that no more refining is necessary, we perform aconservative refinement and compute contact information for thechildren of the current node of the BVTT. The contact informationis interpolated between the two levels.Naturally, LOD interpolation increases the number of nodes ofthe BVTT that are visited. However, for complex models and/orcomplex contact scenarios, the gain obtained from the selectiverefinement still makes sensation preserving simplification significantlyoutperform the exact technique, as presented in Sec. 6.6 Implementation and ResultsIn this section we describe some of the models and experimentswe have used to validate our sensation preserving simplification forhaptic rendering.6.1 System DemonstrationWe have applied our sensation preserving simplification for hapticrendering on the models listed in Table 2. The complexity andsurface detail of these models can be seen in Fig. 9.Models Lower Upper Ball Golf GolfJaw Jaw Joint Club BallOrig. Tris 40180 47339 137060 104888 177876Orig. Pcs 11323 14240 41913 27586 67704Simp. Tris 386 1038 122 1468 826Simp. Pcs 64 222 8 256 64r 1 144.49 117.5 169.9 157.63 216.3r λ 12.23 19.21 6.75 8.31 7.16Free LODs 6 8 3 8 6LODs 15 15 17 16 18Table 2: Models and Associated Hierarchies. The number of triangles(Orig. Tris) and the number of convex pieces (Orig. Pcs) ofthe initial mesh of the models; the number of triangles (Simp. Tris)and the number of convex pieces (Simp. Pcs) of the coarsest LODobtained through simplification; resolution (r 1 and r λ ) of the finestand coarsest LOD obtained through simplification; and “free”LODs and total number of LODs. The resolutions are computedfor a radius of 1 for all the objects.As seen from the results in Table 2, we are able to simplify themodels to LODs with only a couple hundred convex pieces or less.In other words, the sensation preserving selective refinement can beapplied at earlier stages in the contact query, and this allows moreaggressive culling of parts of the BVTT whenever the perceivableerror is small.A79

Figure 9: Benchmark Models. From left to right, moving upper and lower jaws, interlocking ball joints and interacting golf club and ball.With the aforementioned models, we have performed the followingproof-of-concept demonstrations:• Moving upper and lower jaws.• Golf club tapping a golf ball.• Interlocking ball joints.These demonstrations have been performed using our sensationpreserving haptic rendering, a six-DOF Phantom T M haptic device,a dual Pentium-4 2.4GHz processor PC with 2.0 GB of memoryand a NVidia GeForce-4 graphics card, and Windows2000 OS. Ourimplementation, both for preprocessing and for the haptic rendering,has been developed using C++. For the force computation ofthe haptic rendering we have used penalty methods based on thecontact tolerance δ [Kim et al. 2002]. We choose the value of δso that the maximum force of the haptic device is exerted for a 0contact distance with the optimal value of stiffness.6.2 Studies on Perceivable Contact InformationThe performance of the sensation preserving haptic rendering isheavily determined by the selection of the threshold of weightedsurface deviation s 0 . If the chosen value is too high, the perceivedcontact information will deviate too much from the exact contactinformation. On the other hand, if the value is too low and thesimplified models used are moderately complex consisting of morethan a thousand convex pieces, the contact query will no longer beexecutable at the required rate. This severely affects the realism ofhaptic perception.We have designed a scenario where we could test the fidelityof the sensation preserving selective refinement. In this scenario,users can touch the model of the golf ball with an ellipsoid. Theellipsoid has varying curvature, implying the existence of a widerange of contact scenarios, where the selective refinement will stopat varying LODs.12 users experimented with this scenario and reported that theperception of contact information hardly varied for values of s 0 inthe range between 0.025 and 0.05 times the radius of the models.(For readers interested in the detail of experimental data, pleaserefer to Appendix C on the conference proceedings CD).6.3 Performance DemonstrationBased on the value of s 0 obtained from the studies, we have successfullyapplied our algorithm to haptic rendering of object-objectinteraction on the benchmarks listed in Sec 6.1. We have alsoperformed an analysis on contact forces and running time for thedemonstrations previously mentioned. We have compared forceprofiles and statistics of the contact query of interactive hapticdemonstrations with offline executions, using smaller error tolerancesand an exact method. By exact, we mean that the distancecomputation for force display is accurate [Ehmann and Lin 2001].In particular, Fig. 10 shows the contact profile, including the forceprofile, the query time and the size of the front of the BVTT, for200 frames of the moving jaws simulation. Fig. 11 shows the contactprofile for 300 frames of the simulation on the golf scene. Thecontact profile of interlocking joints is quite similar to that of theinteracting golf club and golf ball, thus omitted here.For both scenarios, the simulation with s 0 < 5% of the radii ofthe models has been performed in real time, using a haptic device tocontrol the motion of the upper jaw and the golf club respectively,and to display the contact forces to the user. The trajectories of theupper jaw and the golf club are recorded and played back to performthe rest of the simulations offline, since the exact method wastoo slow to be used to keep up with the force update. As shown inthe figure, we observed a gain of two orders of magnitude in thequery time between the interactive haptic rendering using our approachand the exact offline simulation. Note that the spikes in thecontact query time present in Fig. 10 and Fig. 11 result from lack ofcoherence in the traversal of the BVTT. As reflected in the graphs,the query time is more susceptible to lack of coherence when theerror tolerance is lower.Using our approach, the force profiles of simulations with varyingerror tolerances less than 5% of the radii of the models exhibitsimilar and sometimes nearly identical patterns as that of the originalmodels. This resemblance validates our hypothesis on the hapticperception of contacts, inferred from human tactual perception.43210 20 40 60 80 100 120 140 160 180 2001e6 CONTACT QUERY (µs) 2.5%1e50.25%0.025%1e40.0025%exact1e30 20 40 60 80 100 120 140 160 180 2001e5 BVTT FRONT (no. nodes)1e41e3CONTACT FORCE (N)1e20 20 40 60 80 100 120 140 160 180 200Figure 10: Contact Profile for Moving Jaws. Top: The profiles ofthe contact forces displayed using simplification, with varying errortolerances up to 2.5% of the radii of the jaws, all show very similarpatterns. This similarity implies that the sensations of shape providedto the user are nearly identical. Middle: A log plot of contactquery time using simplification with various error tolerances showsup to two orders of performance improvement. Bottom: The numberof nodes in the front of the BVTT is also reduced by more thana factor of 10.A80

32100 50 100 150 200 250 3001e6 CONTACT QUERY (µs) 3%1e50.3%0.03%1e40.003%exact1e30 50 100 150 200 250 3001e5BVTT FRONT (no. nodes)1e41e3CONTACT FORCE (N)1e20 50 100 150 200 250 300Figure 11: Contact Profile for Golf Scene. Top: The profiles ofthe contact forces displayed using simplification, with varying errortolerances up to 3% of the radius of the ball, show nearly identicalpatterns. Middle: A log plot of contact query time using simplificationwith various error tolerances shows more than two ordersof performance improvement. Bottom: The number of nodes in thefront of the BVTT is reduced by nearly a factor of 100.7 Discussion and AnalysisIn this section, we compare our approach with previous work inrelated areas and analyze the various algorithmic and performanceissues.7.1 Comparison with Related WorkMesh Decimation and Filtering: The construction of our multiresolutionhierarchy can be compared with both mesh decimationtechniques and mesh filtering techniques. Representations obtainedthrough these techniques might present better results than our hierarchiesin certain aspects.LODs created using our filtered edge collapse operation willhave a larger surface deviation than LODs of traditional mesh decimation.This deviation inevitably results from combining decimationand filtering. In our framework, the detail at high resolutionis filtered independently of its magnitude, while mesh decimationtechniques will preserve detail to minimize the surface deviation.The elimination of the detail has beneficial consequences in thecreation of the BVH and does not reflect on the output qualityof the haptic rendering, since the filtered detail is quantified andtaken into account in the sensation preserving refinement. Besides,multiresolution representations obtained through mesh decimationtechniques are not valid by themselves to perform efficient contactqueries.Representations obtained through filtering appear smoother thanour representations. The reduction in visual smoothness occurs becausewe use fewer samples (i.e. vertices) to represent meshes withthe same frequency content. This approach is advantageous for ourapplication, because it accelerates the contact queries. In addition,we have also presented a definition that allows comparing the resolutionof the detail of the objects in contact.Contact Queries for Haptic Rendering: As mentioned earlier,the running time of any contact query algorithm depends on boththe input and output size of the problem. Given two polyhedra,characterized by their combinatorial complexity of n and m polygons,the contact query algorithm can have an output size and arun-time complexity as high as O(nm).The discretized approximation presented by McNeely et al.[1999] can avoid direct dependency on the input size of the problemby limiting the number of points sampled and the number of voxelsgenerated. However, its performance on complex contact scenarioswith many collisions is unknown. Both approaches by Gregoryet al. [2000] and Kim et al. [2002] are limited to relatively simplemodels or modestly complex contact scenarios and do not scalewell to highly complex object-object interaction.In contrast, our approach, by reducing the combinatorial complexityof the input based on the contact configuration at each localneighborhood of (potential) collisions, automatically decreases theoutput size as well. In addition, its selection of LODs is contactdependentto minimize the perceived difference in force display,while maximizing the amount of simplification and performancegain possible. This method is perhaps the first “contact-dependentsimplification” algorithm for collision queries as well.7.2 Generalization of the Algorithmic FrameworkDue to the hard time constraints of haptic rendering, we have chosena collision detection algorithm using BVHs of convex hulls andautomatic convex surface decomposition. The choice of collisiondetection algorithm imposes convexity constraints on the simplificationprocess and the hierarchy constructionThese constraints are rather specific. However, the algorithmicframework for generating the multiresolution hierarchy for sensationpreserving contact queries that we have developed and presentedin this paper is general and applicable to other collision detectionalgorithms.Furthermore, although we focus on contact determination forhaptic rendering in this paper, our approach for sensation preservingsimplification can also be applied to other types of proximityqueries, such as penetration depth estimation. Our approach canbe generalized to multiresolution collision detection by automaticallyidentifying superfluous proximity information and thus cleverlyselecting the appropriate resolutions for performing the queriesat different locations across the objects’ surfaces. This key conceptcan significantly accelerate the performance of any proximity queryalgorithm, as we have demonstrated in this paper.A further analysis of the applicability of sensation preservingsimplification to multiresolution collision detection for rigid bodysimulation has been conducted [Otaduy and Lin 2003]. More studyis needed on the relationship between our sensation preserving errormetrics and contact force models of rigid body simulations, but thepreliminary results are promising.7.3 Integration with Graphic RenderingThe LODs selected for haptic rendering are decoupled from the representationof the objects used for visual rendering. This differencein representations can potentially lead to some inconsistency betweenvisual and haptic display, such as the existence of visual gapswhen the displayed forces indicate that the objects are in contact.Future investigation is required for a better integration of multisensorycues in a multimedia environment.7.4 Other LimitationsOur approach can handle triangular meshes with two-manifolds andboundaries. The current implementation is limited to polygonalmodels with connectivity information. As with all simplificationalgorithms that generate levels of detail offline, our approach hasthe similar memory requirement.A81

8 Summary and Future WorkWe have presented a novel sensation preserving simplification toaccelerate collision queries for force display of complex objectobjectinteraction. The resulting multiresolution hierarchy constructedwith a formal definition of resolution enables us to computecontact information at varying resolutions independently fordifferent locations across the object surfaces. By selecting the mostaggressive simplification possible on the fly based on the contactconfiguration, this approach considerably improves the run-timeperformance of contact queries. It makes haptic rendering of theinteraction between highly complex models possible, while producingonly relatively imperceptible changes in the force display.Our approach can also be easily extended to perform time-criticalhaptic rendering while optimizing the fidelity of the force display,using the technique described in [Otaduy and Lin 2003].This new ability to perform force display of complex 3D objectinteraction enables many exciting applications, where hapticrendering of point-object interaction is insufficient. In addition tofurther optimizing and increasing the performance of sensation preservinghaptic rendering, this research may be extended in severalpossible directions. These include haptic display of friction andtextures exploiting our current framework, applications of 6-DOFhaptic rendering to scientific visualization, engineering prototyping,and medical training, as well as formal user studies and taskperformance analysis.AcknowledgmentsThis research is supported in part by a fellowship of the Governmentof the Basque Country, National Science Foundation, Officeof Naval Research, U.S. Army Research Office, and Intel Corporation.We would like to thank Stephen Ehmann and Young Kim fortheir help on integrating SWIFT++ and DEEP with our 6-DOF hapticrendering framework. We are also grateful to Dinesh Manocha,Russell Taylor, Mark Foskey, and the anonymous reviewers fortheir feedback on the earlier drafts of this paper.ReferencesBROOKS, JR., F. P., OUH-YOUNG, M., BATTER, J. J., AND KILPATRICK,P. J. 1990. Project GROPE — Haptic displays for scientific visualization.In Computer Graphics (SIGGRAPH ’90 Proceedings), F. Baskett,Ed., vol. 24, 177–185.CHAZELLE, B., DOBKIN, D., SHOURABOURA, N., AND TAL, A. 1997.Strategies for polyhedral surface decomposition: An experimental study.Computational Geometry: Theory and Applications 7, 327–342.EHMANN, S., AND LIN, M. C. 2000. Accelerated proximity queries betweenconvex polyhedra using multi-level voronoi marching. Proc. ofIEEE/RSJ International Conference on Intelligent Robots and Systems.EHMANN, S., AND LIN, M. C. 2001. Accurate and fast proximityqueries between polyhedra using convex surface decomposition. ComputerGraphics Forum (Proc. of Eurographics’2001) 20, 3.EL-SANA, J., AND VARSHNEY, A. 2000. Continuously-adaptive hapticrendering. Virtual Environments 2000, pp. 135–144.GARLAND, M., AND HECKBERT, P. S. 1997. Surface simplification usingquadric error metrics. In Proc. of ACM SIGGRAPH, 209–216.GOTTSCHALK, S., LIN, M., AND MANOCHA, D. 1996. OBB-Tree: A hierarchicalstructure for rapid interference detection. Proc. of ACM SIG-GRAPH, pp. 171–180.GREGORY, A., MASCARENHAS, A., EHMANN, S., LIN, M. C., ANDMANOCHA, D. 2000. 6-dof haptic display of polygonal models. Proc.of IEEE Visualization Conference.GUIBAS, L., HSU, D., AND ZHANG, L. 1999. H-Walk: Hierarchical distancecomputation for moving convex bodies. Proc. of ACM Symposiumon Computational Geometry.GUSKOV, I., SWELDENS, W., AND SCHRODER, P. 1999. Multiresolutionsignal processing for meshes. Proc. of ACM SIGGRAPH, pp. 325 – 334.HOFF, K., ZAFERAKIS, A., LIN, M., AND MANOCHA, D. 2001. Fast andsimple geometric proximity queries using graphics hardware. Proc. ofACM Symposium on Interactive 3D Graphics.HOLLERBACH, J., COHEN, E., THOMPSON, W., FREIER, R., JOHNSON,D., NAHVI, A., NELSON, D., AND II, T. T. 1997. Haptic interfacingfor virtual prototyping of mechanical CAD designs. CDROM Proc. ofASME Design for Manufacturing Symposium.HUBBARD, P. 1994. Collision Detection for Interactive Graphics Applications.PhD thesis, Brown University.KIM, Y., OTADUY, M., LIN, M., AND MANOCHA, D. 2002. 6-dof hapticdisplay using localized contact computations. Proc. of Haptics Symposium.KLATZKY, R., AND LEDERMAN, S. 1995. Identifying objects from ahaptic glance. Perception and Psychophysics 57, pp. 1111–1123.KLOSOWSKI, J., HELD, M., MITCHELL, J., SOWIZRAL, H., AND ZIKAN,K. 1998. Efficient collision detection using bounding volume hierarchiesof k-dops. IEEE Trans. on Visualization and Computer Graphics 4, 1,21–37.LARSEN, E., GOTTSCHALK, S., LIN, M., AND MANOCHA, D. 2000.Distance queries with rectangular swept sphere volumes. Proc. of IEEEInt. Conference on Robotics and Automation.LIN, M., AND GOTTSCHALK, S. 1998. Collision detection between geometricmodels: A survey. Proc. of IMA Conference on Mathematics ofSurfaces.LOMBARDO, J. C., CANI, M.-P., AND NEYRET, F. 1999. Real-timecollision detection for virtual surgery. Proc. of Computer Animation.LUEBKE, D., AND HALLEN, B. 2001. Perceptually driven simplificationfor interactive rendering. Rendering Techniques; Springer-Verlag.LUEBKE, D., REDDY, M., COHEN, J., VARSHNEY, A., WATSON, B.,AND HUEBNER, R. 2002. Level of Detail for 3D Graphics. Morgan-Kaufmann.MARK, W., RANDOLPH, S., FINCH, M., VAN VERTH, J., AND TAYLORII, R. M. 1996. Adding force feedback to graphics systems: Issues andsolutions. Proc. of ACM SIGGRAPH, 447–452.MCNEELY, W., PUTERBAUGH, K., AND TROY, J. 1999. Six degreeof-freedomhaptic rendering using voxel sampling. Proc. of ACM SIG-GRAPH, 401–408.OKAMURA, A., AND CUTKOSKY, M. 1999. Haptic exploration of finesurface features. Proc. of IEEE Int. Conf. on Robotics and Automation,pp. 2930–2936.O’SULLIVAN, C., AND DINGLIANA, C. 2001. Collisions and perception.ACM Trans. on Graphics 20, 3, pp. 151–168.OTADUY, M. A., AND LIN, M. C. 2003. CLODs: Dual hierarchies formultiresolution collision detection. UNC Technical Report TR03-013.PAI, D. K., AND REISSEL, L. M. 1997. Haptic interaction with multiresolutionimage curves. Computer and Graphics 21, 405–411.REDON, S., KHEDDAR, A., AND COQUILLART, S. 2002. Fast continuouscollision detection between rigid bodies. Proc. of Eurographics (ComputerGraphics Forum).RUSPINI, D., KOLAROV, K., AND KHATIB, O. 1997. The haptic display ofcomplex graphical environments. Proc. of ACM SIGGRAPH, 345–352.SALISBURY, J. K. 1999. Making graphics physically tangible. Communicationsof the ACM 42, 8.TAUBIN, G. 1995. A signal processing approach to fair surface design. InProc. of ACM SIGGRAPH, 351–358.A82

APPENDIX A: Pseudo Code for MultiresolutionHierarchy GenerationCompute surface convex decompositionDump initial LODn = number of convex piecesCompute resolution of edgesInitialize edges as validCreate priority queuewhile Valid(Top(queue)),if FilteredEdgeCollapse(Top(queue)) thenPopTop(queue)Recompute resolution of affected edgesReset affected edges as validUpdate priority of affected edgesAttempt merge of convex pieceselseSet Top(queue) as invalidUpdate priority of Top(queue)endifif Number of pieces ≤ n/2 thenDump new LODn = number of convex piecesendifendwhilewhile Number of pieces > 1,Binary merge of piecesendwhileALGORITHM 0.1: Creation of the HierarchyAPPENDIX C: Studies on Perceivable ContactDetailsIn this appendix we briefly describe an informal user study thathas been conducted to test the fidelity of the sensation preservingsimplification for haptic rendering, and also to help us identify whatare the error tolerances for which the missing surface detail is notperceptible to the users.The scenario of our experiment consists of a golf ball (pleaserefer to the paper for statistics of the model) that is explored with anellipsoid as shown in Fig. 1. The ellipsoid consists of 2000 trianglesand one single convex piece. For simplicity, we have only appliedthe simplification to the golf ball and left the ellipsoid invariant.Thus, the fidelity of the sensation preserving simplification relieson the adequacy of the resolution of the golf ball that is selected ateach contact.APPENDIX B: Overview of the Haptic RenderingFrameworkIn this appendix, we give an overview of our six-degree-of-freedomhaptic rendering framework for displaying force and torque betweentwo objects in contact. We compute the displayed force basedon the following steps:1. Perform a contact query between two objects A and B, collectingthe set S of nodes of the front of the BVTT that are insidea distance tolerance δ, at the appropriate resolution.2. For all nodes ab ∈ S, compute the contact information:(a) If the pair ab is disjoint, compute the distance, the contactnormal and the closest points and features (i.e. vertex,edge or face).(b) If the pair ab is intersecting, compute the penetrationdepth, the penetration direction, and penetration features1 .Figure 1: Scenario of the User Study.12 users have experimented with this scenario. They were askedto identify at what value of the threshold s 0 of the sensation preservingselective refinement they started perceiving a deviation inthe perception of the surface detail of the golf ball. The values of s 0were in the range from 0.05% to 20% of the radius of the ball. InTable 1 we indicate how many users picked each threshold value.The users also reported that the main characteristic that they exploredwas the perceptibility of the dimples of the golf ball.s 0 ≥ 10% 5% 2.5% 1% ≤ 0.5%no. users 0 4 7 1 0Table 1: Results of the User Study.Based on the results from this informal user study, we selectedvalues of s 0 in the range of 2.5% to 5% of the radii of the modelsfor the demonstrations presented in the paper.3. Cluster all contacts based on their proximity.4. Compute a representative contact for each cluster, averagingcontact information weighted by the contact distance.5. Compute a penalty-based restoring force at the representativecontact of each cluster.6. Apply the net force and torque to the haptic probe.1 By the penetration features, we mean a pair of features on both objectswhose supporting planes realize the penetration depth.A83

Proceedings of DETC2004:2004 ASME Design Engineering Technical ConferencesSeptember 28-October 2, 2004, Salt Lake City, Utah USADETC2004/DAC-57507A HAPTIC SYSTEM FOR VIRTUAL PROTOTYPING OF POLYGONAL MODELSDavid E. JohnsonSchool of ComputingUniversity of UtahSalt Lake City, USAdejohnso@cs.utah.eduPeter WillemsenSchool of ComputingUniversity of UtahSalt Lake City, USAwillemsn@cs.utah.eduElaine CohenSchool of ComputingUniversity of UtahSalt Lake City, USAcohen@cs.utah.eduABSTRACTVirtual prototyping attempts to replace physical modelswith virtual models for the purpose of design evaluation. Onetask a virtual prototyping environment can address is theaccessibility of the components of a mechanical system. In thispaper, we demonstrate a haptics-based virtual prototypingsystem for finding collision-free paths for moving models incomplex polygonal environments. The system can handlemodels and environments with hundreds of thousands oftriangles, and augments innate human talents at searching forcollision-free paths.INTRODUCTIONWhile mechanical model design increasingly relies uponcomputer-aided design (CAD) and sophisticated simulationprograms, physical prototypes still play an important role indesign evaluation. Since physical prototypes are expensive tobuild, and may take significant time to manufacture, virtualprototyping environments attempt to replace as muchfunctionality of the physical prototypes as possible with avirtual prototype.Figure 1: GUIDING A GEAR PAST A SPRING PART IN A SAMPLE VIRTUALPROTOTYPING SESSIONAccessibility is a design evaluation task that is difficult tosimulate on a computer. Two main reasons preclude easyautomatic simulation:• Computation of a collision-free path for complexmodels is difficult and time-consuming,• Modeling human manipulation capabilities isdifficult.We propose a haptic system for virtual prototyping that allowshuman guidance and intuition in developing a collision-freepath between virtual models (Figure 1). This type of systemprovides the intuitive usability of a physical prototype, yetretains the cost and time advantages of a computer model.Our system has several advantages over existing hapticsystems:• Exact distances between models are computed, so thesimulation is accurate to the resolution of the models.• Haptic feedback is provided before models collide, sono invalid interpenetration of the models needs tooccur.• Models in the environment are freely movable,providing real-time adjustment to the scene as desired.The virtual prototyping environment is demonstrated on anexample mechanical system with hundreds of thousands oftriangles.BACKGROUNDVirtual prototyping of accessibility tasks is closely relatedto the area of path planning. The main distinction is that invirtual prototyping, there is some assumption of humaninvolvement, whereas path planning is usually more of anautomatic technique.Path planning methods fall into two main categories –global methods and local methods. Global methods try tosample the configuration space of the model and theenvironment, and then connect together collision-free instancesinto a collision-free path[1][2][3]. Local methods use local1 Copyright © 2004 A84byASME

Figure 2: THE SET OF LMDS PRODUCES REPULSIVE FORCES THAT KEEPMODELS FROM COLLIDING.repulsion techniques to avoid collisions, while being drawntowards a distant goal[4]. However, these local methods can getstuck in local minima and never reach the goal. Our hapticssystem is similar to the local path planning approach, but useshuman guidance to push models past local minima.Haptics[5] has been proposed as a virtual prototypinginterface in prior work. Hollerbach et al. [6] computed fastpenetration depths between a point and a spline model to createa sensation of contact with the model, as did researchers at FordMotor company in [7]. Nelson[8] developed a fast techniquefor haptic rendering of two spline models in contact and alsoadapted the method to moving linkages[9]. Ruspini created ahaptic rendering system for point interactions with complex,moving polygonal models[10].McNeely used a six degree-of-freedom (DOF) device tomanipulate a point-sampled model with a large-scale voxelenvironment[11]. The environment in that system is static;however, that approach guarantees a worst case computationtime, important for reliable haptic rendering. They report thatthey were able to use haptics to find collision-free paths incomplex environments for which global path planningalgorithms failed.APPROACHOur virtual prototyping system is based on a 6-DOF hapticrendering method for complex polygonal models[12][13]. Thathaptic rendering method uses techniques for finding localminimum distances (LMDs) between polygonal models andcomputing local updates to the LMDs while the models move.This paper uses those techniques in conjunction with methodsfor maintaining a collision-free path and visualizing the resultto produce a virtual prototyping environment.The basic haptic rendering approach is to find LMDsbetween the moving polygonal model and the environment.Each LMD acts like a spring guiding the moving model awayfrom collision with the environment (Figure 2), while humaninteraction guides the model towards its goal. In order tomaintain haptic rates, the set of LMDs are updated using a localgradient search.These two steps, a search for LMDs and a local update, aresummarized in the next two sections, please refer to the originalpapers for a more complete description.Spatialized Normal Cone PruningThe haptic rendering searches for LMDs using spatializednormal cones[14], which hierarchically encapsulate the positionand spread of surface normals over a model. The algorithm usessurface normal information to find portions of each model thatFigure 3: A SPHERE BOUNDS A PORTION OF POLYGONAL GEOMETRY AT ANODE OF THE HIERARCHY. THE NORMAL SPAN OF THE NODE GEOMETRYIS ENCAPSULATED IN A NORMAL CONE. THE RANGE OF POSSIBLE LINESBETWEEN CLOSEST POINTS BETWEEN NODES IS BOUNDED BY A DOUBLECONE BETWEEN BOUNDING SPHERES. AT EACH PAIR OF NODES, THEALGORITHM CHECKS TO SEE IF THE NORMAL SPANS OVERLAP AND AREALONG A VECTOR CONTAINED WITHIN THE SPAN OF POSSIBLE MINIMUMDISTANCE LINES.Figure 4: THE LOCAL GRADIENT SEARCH CHECKS NEIGHBORING FEATIURESFOR NEW CLOSEST POINTS. WHEN THE LAST POINT IS ON A FACE,NEIGHBORING FACES ARE CHECKED. WHEN THE LAST POINT IS ON ANEDGE, ONLY TWO NEIGHBORING FACES ARE USED. THE LAST CASE IS FORA VERTEX, WHERE ALL THE TOUCHING FACES ARE CHECKED.point towards each other and are collinear with the line betweenpotential closest points on the models, a condition whichrepresents a local minimum distance (Figure 3). Nodes in thehierarchy that cannot form a local minimum distance solutionare pruned, while remaining nodes are subdivided and thealgorithm recursively applied. At leaf triangles, exact testscompute whether a local minimum distance solution exists.Local Search with Gradient DescentEach LMD pair from the normal cone computationrepresents locally closest points on the two models. The LMDpairs are updated at haptic rates by a local gradient searchalgorithm[13]. The gradient search checks neighboring featureson the polygonal model for a new pair of points that are closerthan the current pair (Figure 4). This search repeats for eachLMD until the LMD converges to a new distance.Combined Search and UpdateAs fast as the global normal cone search can computeLMDs, it introduces new LMDs and deletes those that are nolonger needed. In our examples, this update happened at 10-100Hz. The local gradient search then updated the LMDs atover 1000Hz for stable haptic rendering.Force and TorqueEach LMD pair within a cutoff distance contributes to thetotal force and torque being applied to the model under thecontrol of the force-feedback device. We approximate thecenter of mass and moments of the moving model using anoriented bounding box approximation. These forces and torquesare reflected back to the user by the 6-DOF force-feedbackdevice.2 Copyright © 2004 A85byASME

is always valid, and if the moving model can reach its goal, theproblem has been solved.Detecting CollisionsCollisions are detected when the smallest LMD falls belowan adjustable parameter. This parameter can represent error inthe fit of the polygonal model to an original CAD model, or itcan represent a desired minimum clearance between models.Detecting collisions in this fashion, instead of with actualmodel intersection, provides more control over simulationaccuracy.Figure5: THE 6-DOF PHANTOM.SYSTEM DESIGNOur virtual prototyping system is based on a Sensable sixDOF PHaNTOM haptic interface (Figure 5). The computationsrun on a dual processor Pentium 4 2.4 GHz Linux computerwith a gigabyte of RAM and a GeForce 4 Ti 4400 graphicscard.The software architecture uses a multi-threaded design tohandle the different rates of computation needed for graphics,global normal cone search, and the local updates with forcecomputations. The application uses three threads: a globalsearch thread, a local update thread, and a graphics thread. Thisarchitecture allows us to restrict the computational load of thegraphics and global threads, and let the local update run as fastas possible. On a two-processor system, this translates into thelocal update getting one processor to itself and the other threadssharing the second processor.VIRTUAL PROTOTYPING SYSTEMSince we compute LMDs while the moving model is stillsome distance from the environment models, haptic forces areused to guide the moving model away from collision with theenvironment (Figure 6). The onset distance for forces isadjustable, so the user can decide how much clearance betweenmodels is desired during testing. In general, the LMDs tend toapproximate the local distance field between the models, andthe forces tend to push the moving model towards the medialaxis between the models. Since the medial axis is the surface ofmaximum clearance between models, these forces tend to guidethe moving model towards the safest path.Collision-Free PathWhile the test object is being moved by the hapticinterface, its position and orientation are stored in a buffer. Thisbuffer allows the motion of the test object to be played back forreview, analysis, or further modification.If the moving model is forced to penetrate an environmentmodel by the user, the simulation is no longer valid. A collisionstate is detected and the simulation is rolled back, using thestored positions and orientations in the buffer, until the modelstate is valid. The simulation can then resume, and the user cantry new approaches for finding a collision-free path. Thismeans that the path stored by our virtual prototyping programPath VisualizationSince we store model positions and orientations during thesimulation, a sampling of the path of the model can bevisualized. Drawing a copy of the moving model at eachsampled location (or some subset), allows the user to check thevalidity of the collision-free path, and to examine any unusualmaneuvering needed to safely guide the model. One drawbackis that if many positions are visualized simultaneously, theframe rate of the display can slow.InterfaceThe main interface is the 6-DOF haptic interface. Afterloading models into the environment, the position of thecurrently selected model is controlled by the user moving thehaptic interface. The selected model is changed with keyboardcommands, so any model in the environment is freely movableby the haptic interface.Keyboard commands also control the recording of thecollision-free path, stopping of recording, and visualization ofthe path in playback mode.The current set of LMDs are displayed as red lines betweenthe two models. They help provide feedback cues to the relativepositions of the two models in the absence of stereo viewing.Figure 6: THE LMDS PROVIDE GUIDANCE IN REGIONS OF LIMITEDCLEARANCE.3 Copyright © 2004 A86byASME

Figure 7: HAPTICS GUIDES THE CRANK MODEL THROUGH THE HOLES WHILE AVOIDING THE TEAPOT MODEL. THE FINAL PATH IS VISUALIZED AS A SAMPLING OFMODEL POSITIONS DURING THE SIMULATION.RESULTSWe tested our virtual prototyping system with a variety ofmodels. In the tests, we threaded one moving model around andinside the environment model. The tests were such that themoving models needed to be oriented properly to fit throughthe gaps in the environment model, so that success withouthaptic feedback would be difficult. The model and environmentsizes ranged from around 6,000 triangles to 113,000 triangles.In all the cases, we were able to intuitively find a collision-freepath to accomplish the goal.Helicopter-RockerThe final test used a helicopter model with 113,000triangles and a rocker arm model with 40,000 triangles. For thistest, we wanted to pass the rocker through the open window ofthe helicopter, around the interior, and back out, avoiding otherstructures such as the helicopter blade and tail (Figure 8).The haptic rendering system was able to provide usefulfeedback during this test, creating a collision-free path underuser guidance.Gear-Spring PartIn the first test, we used a gear model with 6,300 trianglesand a spring part model with 23,500 triangles. The goal was tohave the gear enter the spring, traverse down the body, and thenexit the spring. There was limited clearance between the gearand the spring and spring body, as well as between the coils ofthe spring. The gear model had to be turned almost flat to fitthrough the coils. However, with haptics, the forces naturallyguided the model (Figure 6) along a safe path (Figure 1).Crank-Holes-TeapotIn this example, we used a crank model with 45,000triangles, a three-holed block with almost 12,000 triangles, anda teapot model with 5,600 triangles. We used the hapticinterface to position the block and the teapot in such a way thatthere was not a clear path from one hole to the next. The goal inthis test was to thread the crank model through all three holeswhile avoiding the teapot.The haptic interface provided enough cues to the user tofind a path out of the middle hole and to tilt around the teapot,even though that portion of the path was occluded by the teapotduring the test (Figure 7 shows the view during the test and atilted view after the test for the path visualization).Figure 8: THIS TEST INVOLVED MODELS WITH OVER 150,000 COMBINEDTRIANGLES.4 Copyright © 2004 A87byASME

DISCUSSION AND CONCLUSIONThe presented system advances the state-of-the-art inhaptically-enhanced virtual prototyping systems by allowingvirtual prototyping on general, freely positioned, polygonalmodels. In addition, our mechanism for rolling back modelcollision states to a collision-free position and orientationsimplifies motion planning by always storing a safe path. Theuse of adjustable distances for both force onset and collisiondistance also enhances the capabilities of the system bysimulating different clearance constraints during the task.Comparison with other systems is difficult. Most otherhaptic rendering methods for polygonal models depend onpenetration to generate forces, which would invalidate thesimulation results. The highest-performing system[15] tocompare with can render similarly sized models as our systembut uses potentially low resolution levels of details incomputing forces. In contrast, we are able to compute exactdistances between high-resolution models and generate forcesbefore penetration.The tests demonstrated the system on a variety of modeltypes and sizes. While the chosen models were not solving anactual real-world problem, the model shapes, resolutions, andtask types are representative of the kinds of problems thesystem can solve.The locality of the LMD computation means that theseenvironments can scale to very large number of triangles, sincemost of the triangles will not come into consideration at anyone time.We hope to improve the power of the virtual prototypingsystem by continuing to improve the speed of the hapticrendering sub-system, by including other measures besidesdistance as a way of generating forces (such as clearance), andgiving feedback to the user indicating the quality of the path asit is being generated.ACKNOWLEDGMENTSThe authors would like to acknowledge support in partfrom the following grants: NSF DMI9978603, NSF CDA-96-23614, and ARO DAAD 19-01-1-0013.REFERENCES[1] J.F. Canny. The Complexity of Robot MotionPlanning. ACM Doctoral Dissertation Award. MITPress, 1988.[2] L. Kavraki, P. Svestka, J. C. Latombe, and M.Overmars. “Probabilistic roadmaps for path planningin high-dimensional configuration spaces,” IEEETrans. Robot. Automat., pages 12(4):566–580, 1996.[3] M. Foskey, M. Garber, M. Lin, and D. Manocha, “AVoronoi-Based Hybrid Motion Planner”, Proc.IEEE/RSJ International Conf. on Intelligent Robotsand Systems, 2001.[4] O. Khatib. “Real-time obstacle avoidance formanipulators and mobile robots,” in IJRR, 5(1):90–98,1986.[5] K. Salisbury et al., “Haptic rendering: programmingtouch interaction with virtual objects”, in Symp. onInteractive 3D Graphics, 1995. pp. 123-130.[6] J. Hollerbach et al., “Haptic interfacing for virtualprototyping of mechanical CAD designs,” ASMEDesign for Manufacturing Symposium, (Sacramento,CA), Sept. 14-17, 1997.[7] P. Stewart et al., “CAD Data Representations ForHaptic Virtual Prototyping”, Proceedings ofDETC’97, 1997 ASME Design Engineering TechnicalConferences, Sept. 14-17, 1997, Sacramento,California.[8] D. Nelson, D. Johnson, and E. Cohen, “HapticRendering of Surface-to-Surface Sculpted ModelInteraction,” in Proc. 8th Annual Symp. on HapticInterfaces for Virtual Environment and TeleoperatorSystems, (Nashville, TN), ASME, November 1999.[9] D. Nelson and E. Cohen., “Optimization-BasedVirtual Surface Contact Manipulation at Force ControlRates,” in IEEE Virtual Reality 2000, (NewBrunswick, NJ), IEEE, March 2000.[10] D. Ruspini, K. Kolarov, and O. Khatib, “The HapticDisplay of Complex Graphical Environments,” inComputer Graphics Proceedings, SIGGRAPH 1997.Aug. 3-8. pp. 345-352.[11] W. McNeely, K. Puterbaugh, and J. Troy. “Six degreeof-freedomhaptic rendering using voxel sampling”.Proc. of ACM SIGGRAPH, pages 401–408, 1999.[12] D. Johnson and P. Willemsen, “Six Degree-of-Freedom Haptic Rendering of Complex PolygonalModels,” in Proc. 2003 Haptics Symposium , 2003.[13] D. Johnson and P. Willemsen, “Accelerated HapticRendering of Polygonal Models through LocalDescent,” in Haptics Symposium 2004, IEEE, 2004.[14] D. Johnson and E. Cohen, “Spatialized Normal ConeHierarchies,” in Proc. 2001 ACM Symposium onInteractive 3D Graphics, Research Triangle Park, NC,March 19-21, 2001. pp. 129-134.[15] M. Otaduy and M. Lin, “Sensation PreservingSimplification for Haptic Rendering”, in Proceedingsof ACM SIGGRAPH 2003/ACM Transactions onGraphics, Vol. 22, pp. 543-553, San Diego, CA. 2003.5 Copyright © 2004 A88byASME

šŠP›1/n=?PO¡e¢£ˆ¤5¤-¥‡¦.£‚¢§ž+‡£e¨&¢—©,¤}¦¤žVª«ˆ©,¢—©, Oª¬ ¤­y£`®(©—¦e¢—¥J£ˆ¯nŸd («¨ž-¯n©—¡œJžŸdH£!#"%$&'()¥+*,'-§¡.¤$¥¨§©¢¡¤£¦¥¨§©§££/%10324*,'-§¡5&$'-6£!$78092;:

ÔOÚVÒ-âÚVÝyÖ)âJßÃÞOÙ àAÔã Ö6ÞOâ!ådÐ1ÑÒdàÒVã ÑJÞ‡ÜuÙ Ò1F¨äÖCÙ Ò-Õnã¤Ù5ÔOÚ5ê‡Ö)âØuÔÛ)ÞˆØOÖ)Ú(ÔOÛ;ÔˆÜ1Þ(áJÖ)àAÔ“Û¢Õ¤äJÙ¤ß,Ô“Ú-Òuã Þ+Ô“Û)Û6Þ(ç ã¤Ù5ÔOÚVÖ6âJØ}ÔOÚeÙ ÞˆÕ¤ÕÚ-Û)ÞOÕ¤Ò-Õ@ã·ÓÞˆÖ)â¨ãÞˆâ|Ò(ÔOÚ5Ñ|Ó©ÙIcP280 Íz·2;›8zA*Ì2ÑÔ“ÓJã Ö)ÚXׇÖ)Ù¤ã äÔOÛMÒ-â‡×‡ÖCÙ ÞˆâàÒVâ‡ãdàcäÕ@ãdàÒ-ÒVãdã ÑJÒuäÓ&ÜJÔ“ã Ò»Ù5Ôã Òý0 +.-6 *,4 08 0 4 9=:8 *54 9;ÔOâÜ+ãÒ-Õ@ã Ö)âØ©ëJß—ÔOÕ@ãMäÓ&ÜJÔ“ã ÒlÞOߦØOÛ)ÞˆéÔ“ÛÔ“âÜAÛ)Þ¨Ú(ÔOÛ!Ú-Û)ÞˆÕ¤Ò-Õ@ãMÓÞOÖ6â¨ã?Ô“ÓÓJÙ Þ(á©Ö)àAÔ“èãÖ)ÞˆâÕ-ëMÚVÞˆàÓä©ã5Ô“ã Ö)ÞˆâÔOÛ)ÛCÝ`Ò!GÚ-Ö)Ò-â¨ã·Õ¤äJÙ¤ß,Ô“Ú-ÒXÒ-×OÔOÛ)äÔã Ö)Þˆâ#Ô“âǛ âJÞOÙ àAÔOÛãÖ6ÞOâXã Ò-ÚeÑJâÖ+F¨äÒ-Õ-ëÔOâÜuÕ¤àÞ¨Þ“ã ÑÆÒGÚ-Ö)Ò-â¨ã?ã¤Ù5ÔOâÕ¤ÖCã Ö)ÞˆâJÖ)âØ[ÔOÚVÙ ÞOÕ¤ÕÒ-×OÔOÛ)äÔãÖ)Ò-Õ-åQÐ1ÑÖ)Õ1Ö)âã äJÙ âAÛ)Ö)àÖCã Õ8éÞOã Ñã ÑJÒlÚ-ÞOàÓÛ)ÒVá©ÖCã.ÝdÔOâÜÕ¤ä©Ù¤ß,ÔOÚVÒŠéÞˆäJâÜJÔ“ÙàÞ‡ÜJÒ-Û)Õ¢ÓÞOÓäÛ6Ôã Ö6âJØcã ÑÒÒVâ©×‡ÖCÙ ÞˆâJàÒ-â¨ã(åâ‡äàxéÒVÙ1Þ“ßÖ)ààÒ(Ü·Õ¤ä©Ù¤ß,Ô“Ú-ÒxéÞOäâÜÔ“Ù Ö)Ò-Õ-å1óäJÙ¤ã ÑJÒVÙ(ë©ç1ÒÓJÙ ÒVÕ¤Ò-â¨ãlÒeá©ã ÒVâÕ¤Ö)ÞˆâJÕlÖ)âJèã¤ÙÖ)Þˆâ!ëÜ©Ö)àÒ-âÕ¤Ö)ÞˆâJÒ(ÜXÓJÙ ÞOéÒ-Õ-ëÔOâÜÆàxäÛCã Ö)ÓÛ)ÒÚ-Û)äÜ©Ö6âJØ[àއܩÒ-Û¦àAÔ“âÖ)ÓäJÛÔãÞOéÒyÚVÞˆâ¨ã5ÔOÚVã(å Ð1ÑJÒ-Õ¤ÒÔ“Û)؈ÞOÙ ÖCã ÑJàÖ6ÚyÙ Ò-Õ¤äÛCã Õ[Ô“Ù ÒÆã Ò-Õ@ã Ò(ÜTçMÖCã ÑÖ)â9ÔÓJÙÒuÕ@Ý©Õ@ã Ò-à²ã ÑÔ“ãAÖ)â¨ã Ò-ØOÙ5Ôã Ò-Õ·ÔyÙ Ò-Õ¤Ò(ÔÙ ÚeÑ`àÞ‡ÜJÒ-Û)Ö)âØGÓÔOÚ5êOÔO؈ÒOëÚ-ÞˆàÓJÛ)ÒVãÞׇÖÜ©Ò·ÔÆÚVÛ6ÞOÕ¤Ò(Ü}ß—ÞOÙ àoÕ¤ÞˆÛ)ä©ã Ö6ÞOâ|Ö)â+ã ÑJÒÿlÒ-Û)Õ¤ÞˆârÒVãcÔOÛ,åMì@î(ïOïˆï¨ñÁÓJÙäJÙ Ò(ÜÆÿ£ ¡¢£¤¢öXàÞ‡ÜJÒVÛ6ÕŠäÕ¤Ö)âØXÔׂÒVÛ6Þ¨ÚVÖ)ã´ÝGÜJÞˆàAÔ“Ö)âyßÃÞOÙÁã¤Ù5ÔOÚ-Ö)âJØuÕ¤Ú-äÛ)Ó©ãîì¢?Ö6ÒVÕ¤Ò-âJßÃÒ-Û6Üëî(ï¡ Oï©õC¢lÖ)Ò-Õ¤Ò-â©ß—Ò-Û6Ü&ë!î-ïˆï¡¢‚ñ;çMÖ)ã Ñ[éÞOã Ñ[ÔöJÔÙ Ú-ÞˆÕýlÛ)ÓÑÔÞˆäJÕdýM٠णuÔ“Õ@ã ÒVÙX즥¨Ô“Ú-ÞˆéJÕ¤Ò-â|ÒeãÔOÛ,å)ëŠî-ïˆï¨§¨ñxÔ“âÜrÔ ¥ ÑÔ“â¨ã ÞˆàþnÒVá‡ã¤ÙÛ)Þ¨Ú(Ô“Û6ÛCÝ9Ú-ÞOâ‡×‚ÒVáTÒVâÜ©èÌÒVú&ÒVÚVã ÞOÙ»àÞ‡ÜJÒ-Û,åøTÑÖ)Û)Òyã ÑJÖ6Õ[àÒVã ÑÞ‡Ü#Ô“Û)Û6Þ(çMÕÑJÒxÒ-âÜ©èÌÒVú&Ò-Úeã ÞOÙŠã Þ·éÒcÕ¤ÚVäÛ)ÓJã ä©Ù Ò(Üëã ÑJÒcÚ-ÞOâ‡ã5Ô“ÚVãàއܩÒ-Û¦Ö)ÕÁÔ·Õ¤Ö)âJ؈Û)ÒãÑÒlÔOÓJÓJÙ ÞˆÔOÚ5Ñ[Ö)Õ1Õ¤ä©Ù¤ß,ÔOÚVÒnÞOÙ Ö)Ò-â¨ã Ò(ÜçMÖCã Ñ[âJÞ%ã¤Ù5Ô“âÕ¤ÖCã Ö)ÞˆâÖ)âJØÓÞOÖ6â¨ã1ÔOâÜãÑÔ“ÓJã Ö)Únß—Ò-Ò-ÜJéÔ“Úeê·Ü©Ò-ׇÖ6ÚVÒì©£uÔOÕ¤Õ¤Ö)Ò%ÔOâÜuöJÔOÛ)Ö)Õ¤éä©Ù¤Ý¨ë&î(ïˆï‡ñ5åÔ“Û6ØOÞOÙ ÖCã Ñàuåš?/n=c›MÍ»ÏcDAÎÖ)ÚÙ Ò-âÜJÒVÙ Ö)âØAÕ@Ý©Õ@ã Ò-àÐ1ÑÒ%؈ވÔOÛ¦Þ“ß8ÔÑÔOÓJãÖ)Õlã ÞØOÒ-âÒVÙ5Ôã Ò%ßÃÞOÙ Ú-ÒVÕÑÔãÚ(ÔOâyéÒdÔ“ÓÓJÛ6Ö)Ò(ÜXã ÞuÔAäÕ¤ÒeÙ ÕÑÔOâÜÆÞ“Ù%Ô“Ù à{ã Þ»Ô“Ú-Ú-äJÙ5Ôã Ò-ÛCÝyÓJÙ ÞOèãÑdÔ?שÖCÙ¤ã äÔOÛ‡àÞ‡ÜJÒ-Û,å¦Ð1ÑÒ-Õ¤Ò1ßÃÞOÙ Ú-Ò-Õ-ë‚Ú(ÔOÛ)Û)Ò(ÜÜJäJÚ-Ò1ÔŠÕ¤ÒVâÕ¤Ò1Þ“ßÚ-ÞOâ‡ã5Ô“ÚVãçMÖCãÕÁÞOß8ã ÑÒdׇÖ)Õ¤äÔ“Û;Ô“âÜyÑÔOÓ©ã Ö)ÚAÜJÖ)Õ¤ÓÛ6Ô-Ý©Õ-ådø#ÑÖ)Û)ÒÖCãÁàAÔ-ÝÆéÒÚ-ÞOâÕ@ã¤Ù5Ô“Ö6â¨ãÑJÒlׇÖ)Õ¤äÔ“Û&ÜJÖ)Õ¤ÓÛ6Ô-Ýã Þ%äÓ&ÜÔã ÒŠÔã1ã.ç8Ò-â¨ã.ÝdßÙ5ÔOàÒ-Õ8ÓÒVÙÔ“Ú-Ú-Ò-Ó©ã5ÔOéÛ)Ò?ß—ÞOÙ¢ãÒ-Õ@ã ÞOÙ Ö)âJØŠß—Þ“Ù Ú-Ò-Õ-ë‚Ù Ò-Õ¤Ö)Õ@ã ÓÒ-âJÒVã¤Ù5Ô“ã Ö)ÞOâcÖ)â¨ã Þlã ÑÒ;ׇÖ)Ù¤ã äÔOÛ‡àÞ‡ÜJÒVÛJÔ“âÜ%Ô“Ù ÒÙÒ(ÜcäJÕ¤Ö)âØÔMç¢ÔOÛ)ÛJàÞ‡ÜJÒ-Û,åÐ1ÑJÒ-Õ¤Ò1Ù Ò-Õ¤ÓÞOâÕ¤Ò¢àÞ‡ÜJÒVÛ6ÕÞ“ßÃã Ò-âcÑÔ(ׂÒÚ(ÔOÛ)Ú-äJÛ6Ô“ãÑJÒÑÔOÓ©ã Ö)ÚAÜJÖ)Õ¤ÓÛ6Ô-ÝàxäÕ@ãàAÔ“Ö)â‡ã5Ô“Ö)âÙ5Ô“ã Ò-ÕÁÞ“ßMÕ¤Ò-ׂÒVÙ5Ô“Û8чäâ©èÕ¤Ò-ÚVÞˆâÜ&ëãÒ(Üxò¡H1ß—ÞOÙQã ÑÒ8שÖCÙ¤ã äÔOÛ©Õ¤äJÙ¤ß,Ô“Ú-Ò-ÕQã ÞŠß—Ò-ÒVÛÕ¤ÞˆÛ)Ö6Üå ¤;ÞOã Ñcã ÑJҢשÖ)Õ¤äÔOÛJÔOâÜ܇ÙÒVÕ@ã ÞOÙ Ö)âØ%ß—ÞOÙ ÚVÒŠÓJÙ ÞOÓÞOÙ¤ã Ö)ÞˆâÔOÛã ÞÁã ÑÒ?ÓÒ-âÒVã¤Ù5Ôã Ö)Þˆâ[ÜJÒ-Ó©ã ÑyìÌü8ÞOÛ)Ø‚Ô“ã ÒÔÁÙ¤;Ù Þ(çMâ!ë¦î-ïˆï¨¨ñlÔOâÜ»Ö)âuã ÑÒ%ÜJÖCÙ Ò-Úeã Ö6ÞOâXÞOß ã ÑÒÁÕ¤ä©Ù¤ß,ÔOÚVÒxâJÞOÙ àAÔOÛ!Ô“ãÔOâÜÖ)ڢܩÖ)Õ¤ÓÛ6Ô-ÝcÖ)âÜ©Ö)שÖ6Ü©äÔOÛ)ÛCÝ%ã5Ô“ácÔŠÕ@Ý©Õ@ã Ò-àuëOã ÑJÒVÙ ÒVßÃÞOÙ Òˆë‚ç8ÒMÜJÖ)Õ@ã¤Ù Ö)éä©ã ÒÑÔOÓJãÑJÒ[¡ ©—Ÿc¥©¯C£‚¢—©— ˆªu¨¦@ (¤5žV¡ ¡Ô“âÜ©£e¨&¢—©—¤%¨¦.(¤ežV¡ ¡ŠÞˆâ¨ã ÞuÜ©Ö)ú&ÒeÙ Ò-â¨ãÁÕ¤ÓÒ-Ú-Ö6ÔOÛãÑÒAÚ-ÞOâ‡ã5Ô“ÚVãxÓÞˆÖ)â¨ã(ëã.Ý©ÓÖ)Ú(Ô“Û6ÛCÝ+ÔuÛ6Þ¨Ú-ÔOÛ¢Ú-Û)ÞˆÕ¤ÒVÕ@ãdÓÞˆÖ)â¨ã(åyýlâ+ÔOÚ-Ú-ä©Ù5Ô“ã ÒãÚ-ÒŠÚ-ÞOàÓä©ã5Ô“ã Ö)Þˆâ·ÜJÒVÓÒ-âÜ©Õ;ÞOâ»Ô؈ިއÜã¤Ù5ÔOÚ5ê‡Ö)âØxÔOâÜdã¤Ù5Ô“Ú-Ö)âØ%ÔOÛ)؈ޓèß—Þ“ÙÓÞˆÕ¤ÒcÚ-ÞˆàÓJäJã ÒVÙ Õdì§Ð1ÑJÞˆàÓJÕ¤ÞˆâyÒVã%ÔOÛ,å)ë1î(ïOï‚÷“Ô©õQÐ1ÑÞOàÓÕ¤ÞOâÒVã%Ô“Û§å)ëÓJäJÙî-ïˆï‚÷é&ñ5åÖCã Ñà Õ¤Ö6âJÚ-ÒMã ÑÒMàAÔOØOâÖCã äÜ©ÒlÔ“âÜÜ©Ö)Ù ÒVÚVã Ö)Þˆâ·ÞOß&ã ÑÒMß—Þ“Ù Ú-ÒŠÜ©Ò-ÓÒ-âÜJÕ8ÞOâÙÑÒŠÔOÚ-Ú-ä©Ù5ÔOÚVÝ·Þ“ßã ÑÒlÚ(ÔOÛ)Ú-äJÛ6Ô“ã Ò(Ü»ÚVÛ6ÞOÕ¤Ò-Õ@ã?ÓÞˆÖ)â¨ãMÔOâÜ·âJÞOÙ àAÔ“Û§å û´àÓÞOÙ¤èãÞˆâJàÒ-â¨ã¢Ù äâJÕMçMÖCã ÑJÖ6â[ã ÑÒÕ¤Ö)àcäJÛÔã Ö)ÞˆâuÓ©Ù ÞOèÐ1ÑJÒàÞ‡ÜJÒ-Û)Ö)âØdÒVâ©×‡ÖCÙê‡Õ@ã5Ô“ã Ö)ÞˆâuçMÑJÖ6Ú5ÑuÑÔOâÜ©Û)Ò-Õ¢ã ÑÒׇÖ)Õ¤äÔOÛÜJÖ)Õ¤ÓÛ6Ô-ÝÚ-ÒVÕ¤ÕlÞOâXÔcØOÙ5Ô“ÓÑJÖ6ÚVÕlç1Þ“ÙÛCݨë‡ã ÑÒŠßÃÞOÙ Ú-ÒÕ¤ÒV٠׈ÞdÛ)Þ¨ÞˆÓ!ëÔOâÜ[ÑJÒ-âÚ-Òlã ÑÒnÚ-Û)ÞOÕ¤Ò-Õ@ãŠÓÞOÖ)â‡ãMÚ-ÔOÛ)Ú-äÛ6Ôèã5ÔOâ¨ãÖ)Þˆâ!ëàxäÕ@ã?Ù äâÆÔ“ãnÕ¤Ò-ׂÒVÙ5Ô“ÛÑ©äJâ܇٠Ò(Ü»ò£H%Ö)âGÞ“Ù5ÜJÒVÙlã Þ[àAÔ“Ö)â‡ã5Ô“Ö)âXÕ@ã Ö)úãàÕ¢Ú-ÞˆàÓJäJã5Ô“ã Ö)ÞOâÕ8ã´ÝJÓJÖ)Ú(ÔOÛ)ÛCÝA؈Û)ÞˆéÔOÛÖ)â»Õ¤Ú-ÞOÓÒˆå ø#ÑÒ-âã ÑJÒÔ“âÜAÓÒV٤ߗޓÙÖ)ÞˆâÆÓJÙ Þ¨ÚVÒ-Õ¤ÕÜJÒeã Ò-ÚVã ÕÔ“âGÔ“ÓÓ©Ù ÞˆÓJÙ Ö6Ôã Òc؈Û)ÞOéÔOÛ Ò-׈Ò-â¨ã(ë&Õ¤äÚ5ÑÆÔOÕÕ¤Ö)àcäJÛ6Ô“ãäÔOÛMÕ¤äJ٤ߗÔOÚ-ÒVÕyì©£»Ö)âJÕ¤ê‡Ý>ÒVã·Ô“Û,å6ëŠî(ïˆï¨§¨ñ5å3Ð1ÑJÖ6ÕAÑJÖ6ØOÑ`äJÓ&ÜÔ“ã Ò·Ù5Ô“ã ÒׇÖ)Ù¤ãÕcã ÑJÒ»Ú-ÞOàÓÛ)ÒVá©ÖCã.Ý}ޓߊã ÑJÒuÔOÛ)ØOÞOÙ ÖCã ÑàÕdßÃÞOÙùâÜJÖ)âØXã ÑJÒ»Ú-Û)ÞˆÕ¤ÒVÕ@ãÛ)Ö6àÖCãÑJÒŠÓJÙ Þ(á©Ö)àÖCã.ÝdÞ“ßã ÑJÒŠÓJÙ ÞOéÒlã ÞcÔ%àއܩÒ-Û,ëJÖCã¢Õ¤Ö)ØOâÔOÛ)Õ¢ã ÑÒlÑÔ“ÓJã Ö)ÚŠÓ©Ù ÞOèãÖ)â‡äÒ-Õdã ÑJÒuÚ-ÞOàÓäJã5Ôã Ö)Þˆâ|çMÖCã ÑTÛ)Þ(ç¢èÌÛ6Ô“ã Ò-âJÚVݨë¢Û)Þ¨Ú(ÔOÛÚ-ÒVÕ¤Õ-ë¢çMÑÖ)ÚeÑ|Ú-Þˆâ¨ãÑÒeÙ Ò-é¨ÝGÑÔOÕÁÜ©Ö6Úeã5Ô“ã Ò(ÜGã ÑJÒcã.Ý©ÓÒ-ÕޓߢàÞ‡ÜJÒVÛ6Õnã ÑÔ“ãÁÚ-ÞOäÛ6ÜÓÞˆÖ)â¨ã%ÔOâÜyãÒ-âÜ©ÒVÙ Ò(Ü&å éÒnÙÑJÞ‡ÜJÕ-åÁÐ1ÑJÒcÑÔOÓJã Ö)ÚxÓJÙ Þ¨Ú-Ò-Õ¤ÕnÙ äJâÕnÞOâXÙ Ò(ÔOÛCè§ã Ö)àÒcàÖ)ÚVÙ Þ¨Ú-ÞOàÓäJã ÒeÙàÒVãàÕÚ-ÞˆàÓJäJã5Ôã Ö6ÞOâÕŠçMÑJÖ)ÚeÑÔ“Ù Òcã´ÝJÓJÖ)Ú(ÔOÛ)ÛCÝyÛ6Þ¨Ú-ÔOÛQÖ6âéÞˆÔ“Ù5ÜJÕÁÔ“âÜÆÓÒV٤ߗޓÙÒäJÕ¤Ö6âJØXÔóÞOÙ%Ö)âÕ@ã5Ô“âÚ-ÒOë&Ö)Û)Û)Ò-ÕxÔOâÜ+öJÔOÛ)Ö)Õ¤éä©Ù¤Ý`ì@î-ïˆï¨§‚ñÔˆÜJ׈ިÚ(Ô“ãÒ-à ã Þ·ã¤Ù5ÔOÚ-ÒcÓÞˆÛCݩ؈ވâÔOÛQàÞ‡ÜJÒ-Û)Õ-åcÐ1ÑÒ-Õ¤ÒcÕ@ÝJÕ@èÚ-ÞˆâJÕ@ã¤Ù5ÔOÖ)â¨ã¤èÌéÔOÕ¤Ò-ÜGÕ@Ý©Õ@ãÕ¤Ú-ÞOÓÒˆåã ÑJÒVÙ âÒeãŠÚ5ÑÔ“ââÒVÛ6ÕŠÚ-ÞˆâJâÒ-ÚVãlã ÑÒã.ç8Þ[Ó©Ù Þ¨Ú-Ò-Õ¤Õ¤Ò-Õlã Þ·ß—Ô“èö‡Ò-ׂÒVÙ5Ô“ÛæÒ-àÕ[Ô“Ù ÒyÞOßÃã ÒVâ3Û)Ö)àÖCã Ò(Ǜ ã Þ}Õ¤Ö)àÓÛ)ÒÆàÞ‡ÜJÒ-Û)Õ[Õ¤Ö6âJÚ-ÒÆã ÑÒÆÑÖ)؈Ñ#ÓÞOÛCÝ©èãÙ Ò1F¨äÖCÙ Ò-Õ ã Þ¨ÞÁàcäJÚeÑxÓJÙ Þ¨Ú-Ò-Õ¤Õ¤Ö)âJØÁã Ö)àÒˆå؈ÞOâÚ-ÞˆäJâ‡ã;ÞOß&ÚVÞˆàÓÛ)ÒVáxàÞ‡ÜJÒVÛ6ÕÒ?ã ÑÒM׈ÔÙ Ö)ÞˆäÕQß—Þ“Ù àÕ8ÞOßÚ-ÞOààcäJâÖ)Ú(Ô“ã Ö)ÞOâÖ6âÞOäJÙ8Õ@ÝJÕ@ã ÒVàuå ¤8ÞOã ÑÚ-Ö)Û)ÖCã5Ô“ãÞ¨Ú-Ò-Õ¤Õ¤Ò-Õ¢ÑÔ׈ÒnÔÁàÞ‡ÜJÒ-ÛàAÔ“âÔOØOÒVÙ1çMÑJÞˆÕ¤ÒnÕ¤ÞOÛ6Òlß—äâJÚVã Ö)Þˆâ·Ö)Õ1ã ÞxàAÔ“Ö6â©èÓ©ÙÒVâÜ·ã ÑJÖ)Õ?ç8ÞOÙ êAã ÞÑÔOâÜJÛ)ÒÛ6Ô“Ù ØOÒVÙnÓÞOÛ)Ý©ØOÞˆâ¢läJÕ¤ÓÖ)âÖÒVãlÔOÛ,åì@î-ïˆïˆ÷‚ñMÒeá©ãÕnÔOâÜ»ã ÞAÓÒVÙ àÖCãŠàÞ“Ù ÒxØOÒ-âÒVÙ5Ô“ÛØOÙ5Ô“ÓÑÖ)Ú-ÕlÓJÙ Ö)àÖCã Ö)ׂÒ-Õ-ë&Õ¤äÚ5ÑÆÔOÕÚ-ÞˆäJâ¨ãÒVâÚVÝÆÔ“ÚVÙ ÞˆÕ¤Õnã ÑJÒxÕ@Ý©Õ@ã Ò-àuålþÔã5ÔAÚ-ÔOÚ5ÑÖ)âØ·ÞˆâÆéÞOã Ñã5Ô“Ö6âyÜJÔ“ã5ÔAÚ-ÞˆâJÕ¤Ö6Õ@ãÑJÒyÕ@Ý©Õ@ã Ò-àuë?Ö)â#Ú-ÞOâ¤äJâÚVã Ö)ÞOâ9çMÖCã ÑTã ÑJÒGÜ©Ö)Õ@ã Ö6âJÚVãXÜ©Ö)שÖ)Õ¤Ö)ÞˆâÕ¤Ö6ÜJÒVÕ»ÞOßÁãÕ?ÔOâÜ·Û)Ö6âJÒ-Õ-ëã ÞcéÒŠã¤Ù5ÔOÚ-Ò-Ü»é¨Ý»ÔcÜJÖ)àÒ-âÕ¤Ö)ÞOâÒ(Ü·ÓJÙ ÞˆéÒOåÓÞˆÖ)â¨ãÖ)éäJã Ö)ÞOâAßÃäâJÚVã Ö)ÞˆâÕ1ÔOâÜ·ö©ÔOÛ)Ö)Õ¤éäJÙ¤ÝýŠÜJÔOÚ5ÑÖ¦ì@î-ïˆï¡¢‚ñQÒ-àÓJÛ6Þ(Ý©Õ;ÜJÖ)Õ@ã¤ÙÑJÒ»ã.ç8ÞÓJÙ Þ¨ÚVÒ-Õ¤Õ¤Ò-Õ-ëMê‚Ò-ÒVÓÕdã ÑÒ[Ú-ÞˆààxäâJÖ6Ú-Ô“ã Ö)ÞˆâÞ“ßnÛ6ÔOéÞ“Ù[éÒeã.ç1ÒVÒ-â|ãÑJÒ(ÔˆÜÆÔãÁÔ·àÖ)âJÖ6àxäàuålÐ1ÑÒxÚ-ÞˆàxéÖ)âÔã Ö)ÞˆâÆÞOß;Ö)âÜJÒVÓÒ-âÜ©Ò-â¨ãŠÓ©Ù ÞOèÞׂÒVÙÞ»ÓÒeÙ àÖ)ã,F‡äÔOÛ)ÖCã.ÝXã¤Ù5Ô“Ú-Ö)âØÔOâÜGЦÔ٤ٷì@î(ïˆïˆ÷‚ñŠäÕ¤ÒxÖ)àÓÛ)Ö)Ú-ÖCãÁÕ¤äJÙ¤ß,Ô“Ú-Ò-ÕãäJÙ Ò-ÜcàÞ‡ÜJÒ-Û)Õ-åòlÞ(ç1Ò-ׂÒeÙ(ëˆã ÑÒMÜ©Ò-Õ¤Ö)؈âÔ“âÜcØ“Ù5ÔOÓJÑÖ)Ú-ÕQÚ-ÞˆààxäJèÞOß&Õ¤Ú-äJÛ)ÓJãÚ-ÒVÕ¤Õ¤Ò-ÕcàAÔáJÖ)àÖ*H-Ö)âØ»Ú-ÞˆàÓJäJã5Ô“ã Ö)ÞOâÔOÛ;Ú(ÔOÓÔOÚ-ÖCã´Ý¨ëÛ)ÞçÚ-ÞˆààxäâJÖ6Ú-Ô“ã Ö)ÞˆâÖ)Ò-Õé¨ÝÆÔOâÜXÛÔ٠؈ÒàއܩÒ-Û¦äÕ¤Ö)âJØ»ÿ£ ¡¢£¤1ö&å&ý?ÕŠÕ¤äÚ5Ñ!ë&ã Þ·äÕ¤Ò%ã ÑÒ-Õ¤ÒâÖCãÑއܩÕ8Ù Ò1F¨äÖCÙ Ò-ÕMÔxÚ-Þˆâ‡×‚ÒVÙ Õ¤Ö)ÞOâ[ßÃÙ ÞOà ã ÑÒlÿ£ £¢¡¤¢öàÞ‡ÜJÒ-ÛÖ)â¨ã ÞcÞOâÒàÒVãÑJÒ(ÔˆÜ&ëÔOâÜyÕ@Ý©âÚ5ÑJÙ ÞOâÖ*H-Ò(ÜÆׇÖ)Õ¤äÔOÛ Ô“âÜÆÑÔ“ÓJã Ö)ÚdÜ©Ö6Õ¤ÓJÛ6Ô(ÝÆÚeÙ Ò(Ô“ã Ò-ÕÁÔÞׂÒVÙÒ(Ô“Û)Ö6Õ@ã Ö)ÚÁÑÔ“ÓJã Ö)ÚׇÖCÙ¤ã äÔOÛÒ-â‡×‡ÖCÙ ÞˆâàÒVâ‡ã(åÙÑÒVÕ¤ÒXÞOã ÑÒeÙ[Ù Ò-Ó©Ù Ò-Õ¤Ò-â¨ã5Ô“ã Ö)ÞOâÕ-åÇÐ1ÑJÖ6Õ·Ú-ÞOâ©×ˆÒVÙ Õ¤Ö)Þˆâ#Ö)Õ[âJÞOã[ÞˆâJÛ)Ý`ÔÞOßÁãÒ(Ü+ã5ÔOÕ¤êéäJãxÚ(Ô“â>éÒ·â‡äàÒVÙ Ö)Ú(Ô“Û)Û)ÝäâJÕ@ã5ÔOéÛ)ÒAÙ Ò-Õ¤äÛCã Ö)âØyÖ)âÚ-ÞˆàÓJÛ)Ö6Ú-Ô“ãDAÏc›lšlÇ0`Í»ÎA2P›Š*Ì03092;ÎlâÖCß—Þ“Ù à ¢ŠÔã Ö)ÞˆâÔ“Û ¤;è.ö‡ÓÛ)Ö)âÒwì,ÿ£ ¡¢£¤1öñ3Õ¤ä©Ù¤ß,ÔOÚVÒ-Õ ÔÙ ÒÿlÞˆâJè:ÒàÞ‡ÜJÒ-Û)Õ?ÜJÒVùâÒ(Ü·é¨ÝAÑÖ)؈ÑuÞ“Ù5ÜJÒVÙMßÃäâÚeã Ö6ÞOâÕ-åÖ)âÔOÚVÚ-äJÙ5Ôãã ÑÒVÙ Õ8ì§ýŠÜÔ“ÚeÑJÖ‚ÒVã¦ÔOÛ,å)ëî(ïOï §¨õ£uÔ“Ù êÒVã¦ÔOÛ,å)ëî-ïˆïˆð‚ñ!ÓJÙ ÞOÓÞˆÕ¤Ò8äJÕ¤Ö)âØÒeÙ àÒ(ÜJÖ6Ô“ã ÒÙ Ò-Ó©Ù Ò-Õ¤Ò-â¨ã5Ô“ã Ö)ÞOâÕã Þ¢Õ¤Ö)àÓÛ)ÖCßÃÝŠã ÑÒã¤Ù5ÔOÚ-Ö)âJØlÞ“ßJÕ¤ÚVäÛ)ÓJã ä©Ù Ò(ÜÖ)â‡ãÒ-àÕÔ“Ù ÒéÞOäâÜGé¨ÝÆâÒVã´ç1Þ“Ù êyÛ)Ö)àÖCã5Ô“ã Ö)ÞˆâJÕ-ë ÔOâÜàÞ‡ÜJÒ-Û)Õ-å·Ð1ÑJÒ-Õ¤ÒÕ@Ý©Õ@ãÒVÕ¤Õ¤Ö6׈ÒÔOÕ%Ô·Ù ÒVÓJÙ Ò-Õ¤Ò-â¨ã5Ôã Ö6ÞOâGßÃÞOÙÑJÖ6ØOÑÛCÝÆÚ-ÞOàÓÔOÚeã%ÔOâÜXݨÒVãnׂÒVÙ¤ÝÆÒVá©ÓJÙ£¢¡¤¢ö»Õ¤äJ٤ߗÔOÚ-Ò%Ö)ÕnÔéÖ)×OÔ“Ù Ö6Ô“ã Ò%ׂÒ-ÚVã Þ“Ù¤èÌ׈ԓÛ6äJÒ(Ü»ÓJÖ6ÒVÚ-ÒVèàއܩÒ-Û)Ö6âJØJå?ý¬ÿ¡çMÖ)Õ¤ÒnÙ5Ô“ã Ö)ÞOâÔOÛß—äJâÚVã Ö)Þˆâ»ÞOßã ÑÒlß—ÞOÙ àÑÒVÙ Òeß—ÞOÙ ÒÆäJÓ&ÜÔ“ã ÒVÕ»ÔÙ ÒyÕ¤Û)Þ(çå¬ø#ÑJÖ)Û6ÒXã ÑJÖ6Õ·Ô“ÓÓJÙ ÞˆÔOÚ5Ñ#ÓÒVÙ àÖCã Õã ÑÒãÕxßÙ Þˆà ÓÞ¨ÞOÙã Ò-àÓÞ“Ù5ÔOÛ1ÓÒVÙ¤èã¤Ù5ÔOÚ-Ö)âJØGÞ“ßnÚ-ÞOàÓÛ)ÒVá+àއܩÒ-Û)Õ-ë8ÖCãÕ¤ä©ú&ÒVÙàAÔOâÚVÒÆÔOâÜ>Ú(ÔOâJâÞ“ã»Ô“Ú-Ú-ä©Ù5Ô“ã Ò-ÛCÝ`Ù ÒVÓJÙ Þ‡ÜJäJÚ-ÒuÕ¤ä©Ù¤ß,ÔOÚVÒ-Õ[çMÖCã ÑTÑJÖ6ØOÑß—Þ“Ù×OÔ“ã ä©Ù Òˆå Ú-äJÙ0 +1-32 *54 076 *,4 0¨8 0 4 9;: ì¦ !#"Jñ%$'&)(*,+.-&)/&)(*,+.-&)/ÔOâÜ4ü8ÞˆÑJÒ-âÐ1ÑÞOàÓÕ¤Þˆâ3ÒVãyÔ“Û§åxì@î(ïOï‚÷“Ô¨ñuÔOÕuç8Ò-Û)ÛcÔ“Õ¥ˆÞˆÑâJÕ¤Þˆâ¨ñ¢ÑÔ(ׂÒÁÜ©Ò-àÞˆâJÕ@ã¤Ù5Ô“ã Ò(ÜuÜ©ÖCÙ Ò-ÚVã?ÑÔ“ÓJã Ö)ÚÙ Ò-âÜ©ÒVÙ Ö)âØÞOß Õ¤ÚVäÛ)ÓJã ä©Ù Ò(Üì@î(ïOï¡àÞ‡ÜJÒ-Û)ÕlÚ-ÞˆâJÕ@ã¤Ù äÚVã Ò(Ü»ßÙ Þˆàäâ¨ã¤Ù Ö)ààÒ(Ü[ÿ¡ £¢¡¤¢ö[Õ¤äJ٤ߗÔOÚ-ÒVÕnäÕ¤Ö)âØ·Ü©Ö)èçMÑJÒVÙ Ò}ã ÑÒ?>2 *,4 0¡@ ßÃÞOÙ à ã ÑÒrÚ-ÞOâ‡ã¤Ù ÞOÛcàÒ-Õ¤Ñëxã ÑJÒ?> 6 *,4 0A@ Ô“Ù Ò}ã ÑJÒç8Ò-Ö)؈Ѩã Õ-ëlÔOâǛ ã ÑJÒB> 8 *54 9 < @ ÔOâÜC> 8 0 4 9 : @ ÔÙ ÒÆã ÑJÒXéÔ“Õ¤Ö6Õ·ß—äJâÚVã Ö)ÞˆâJÕÙ Ò-ÚVãMÓÔ“Ù5ÔOàÒeã¤Ù Ö6Ú?ã¤Ù5ÔOÚ-Ö)âJØJå ø#ÑÖ)Û)ÒnÕ¤Ú-äJÛ)ÓJã äJÙ Ò-Ü·àއܩÒ-Û)Õ1Ú(ÔOâ[éÒlã¤Ù5Ô“Ú-Ò(ÜA90

ì,é&ñ*:+ 8;8ÔyÕ¤ÑJÞOÙ¤ãcÒ-âÞOä؈ÑrãÕnÑJÖ)؈ÑÆäÓ&ÜJÔ“ã ÒÁÙ5Ô“ã ÒˆånýlÛ)Õ¤ÞJë&ã ÑÒ%àÒVã ÑJއܻàcäJÕ@ãŠÓ©Ù Þ‡ÜJäÚVÒàAÔ“Ö6â¨ã5Ô“Ö6âXÖCãÒVß—ÒeÙ·ã ÞÔOÕAÔ>¯C (¤5£ˆ¯¢¤(¯C ˆ¡ežV¡e¢¨© ˆ©—ª¢MÕ¤Ö6âJÚ-Ò»ã¤Ù5Ô“Ú-Ö)âØäJÕ¤Ö)âØ+ÔçMÑÔ“ãdç8Ò»ÙÒ-Õ¤äÛCã Õ¢Ö)â»Õ¤ÒVׂÒVÙ5ÔOÛ&Ú-ÞOàÓÛ)Ö)Ú(Ô“ã Ö)ÞOâÕ?ÔOâÜë‡Ö)â[ß—ÔOÚVã(ëØOÛ6ÞOéÔ“Û!Ú-Û)ÞOÕ¤Ò-Õ@ã?ÓÞˆÖ)â¨ã1ÙàAÔOÛ)ÕAÔOâÜrã5Ô“â؈ÒVâ‡ã Õ-ë¢ÔOÛ)ÞˆâJØçMÖCã Ñ`ÖCã ÕÖ)â‡ã äJÖCã Ö6׈һÚ-ÞOâ‡ã¤Ù ÞOÛMÚeÑÔ“Ù5ÔOÚeèâÞ“ÙÒVÙ Ö)Õ@ã Ö)Ú-ÕMàAÔOê‚ÒlÖ)ã¢Ôx؈ިއܷ٠Ò-ÓJÙ Ò-Õ¤ÒVâ‡ã5Ôã Ö)Þˆâ[ßÃÞOÙ?àÞ‡ÜJÒVÛ6Ö)âJØcÔ“âܻܩÒ-Õ¤Ö)؈âåãÞˆÓÒVÙ¤ã Ö)Ò-ÕlÑÔ(ׂÒÛ)Ò(Ü[ã Þÿ¡ £¢¡¤¢ö·éÒ-ÚVÞˆàÖ)âØdã ÑJÒ«¨ž1­V£‚¤(¢, ÁÖ)âJèÐ1ÑÒ-Õ¤ÒÁÓ©ÙÑÒMÙ Ò-ÓJÙ ÒVÕ¤Ò-â¨ã5Ô“ã Ö)Þˆâ»ÔOâÜ·ÜÔã5Ô%ÒeáJÚ5ÑÔ“âØOÒŠÞOß!؈ÒeèÜJäJÕ@ã¤Ù¤ÝÕ@ã5ÔOâÜÔ“Ù5Üß—Þ“Ù¢ãÒ-â¨ã ÖÔ“Û)Û)Ý[ÒVÙ¤Ù ÞOâÒ-ÞOäÕ¢Ù Ò-Õ¤äÛCã ÕÁì§Ð1ÑÞOàÓÕ¤ÞOâ»ÒVãlÔOÛ,å)ëî-ïˆïˆ÷OÔ¨ñ5åÓÞ“ãÖ)Ú(Ô“Û)Û)Ý}ã¤Ù5Ô“Ú-Ö)âØyÔÆשÖCÙ¤ã äÔOÛ1àއܩÒ-Û8Ú(Ô“â|éÒAé©Ù ÞOèÐ1ÑJÒ»Ô“ÚVãdޓߊÑÔOÓJãÞÆÕ¤Ò-ׂÒVÙ5Ô“ÛMÓÑÔ“Õ¤Ò-Õ-årÐ1ÑJÒ»Õ@Ý©Õ@ã Ò-à²Ú5ÑÒVÚeê‡ÕcÒ-ÔOÚ5Ñ>àއܩÒ-Ûê‚ÒVârÜJÞ(çMâ|Ö)â¨ãÞ(á©Ö6àÖCã´ÝXã Þ»ã ÑÒÓ©Ù ÞˆéÒAÔOâÜ+ÔOÚeã Ö6×OÔ“ã ÒVÕ%ã ÑJÞˆÕ¤ÒàÞ‡ÜJÒVÛ6Õ%Ü©Ò-Ò-àÒ(ÜßÃÞOÙxÓJÙÖ)ÚàއܩÒ-Û)Õ즥QÖ)Ò-ØOÛ¦ÔOâÜ»Ð1Ö)Û)Û)ÒVÙ(ë¦î(ïOï §ˆñ5åÞˆàÒVã¤ÙÖ)ààÒ(Üuÿ£ ¡¢£¤1öXàއܩÒ-Û)ÕÔ“Ù ÒcÚ-ÞˆâJÕ@ã¤Ù äÚVã Ò(ÜXé¨ÝXÚ-ä©ã¤ã Ö)âØ[Ô(ç1Ô-ÝÐ!ÙÖ)ÞˆâJÕ8Þ“ßÔnÿ£ £¢¡¤¢öxÕ¤ä©Ù¤ß,Ô“Ú-ÒläJÕ¤Ö)âØnã¤Ù Ö)ààÖ)âØŠÚVäJ٠ׂÒ-ÕQÖ)âÓÔÙ5ÔOàÒVã¤èÓÞOÙ¤ãÖ)ÚÕ¤ÓÔOÚVÒˆåQû.â·ÞˆäJÙ?Õ@Ý©Õ@ã Ò-à ã¤Ù Ö)ààÖ)âØ%Ö)âJßÃÞOÙ àAÔ“ã Ö)ÞOâ»Ö)Õ¢Ù Ò-ÓJÙ Ò-Õ¤ÒVâ‡ã Ò-Ü»ÔOÕÙÞá©Ö)àAÔOÛ,åTÐ1ÑÒuã¤Ù5ÔOÚ5ê‡Ö)âØ+Ô“Û)؈ÞOÙ ÖCã ÑJàvã ÑJÒ-â|ã¤Ù5Ô“Úeê‡ÕAÔÚVÛ6ÞOÕ¤Ò-Õ@ã·ÓÞˆÖ)â¨ãÓ©ÙÑJÒcàއܩÒ-Û¦äâ¨ã Ö)Û¦Ú-ÞOâ¨ã5ÔOÚVãÖ)ÕnàAÔOÜJÒˆëÔãnçMÑÖ)Ú5ÑGÓÞOÖ)â‡ãlã ÑÒÁã¤Ù5ÔOÚ-Ö)âJØÞOâÆãÖCã Ñàoã5ÔOê‚ÒVÕcÞׂÒVÙ(åyø#ÑÖ)Û)Ò·Ö6â}ÚVÞˆâ¨ã5ÔOÚVã(ëQã ÑÒã¤Ù5Ô“Ú-Ö)âØGÔOÛ)؈ޓ٠Ö)ã ÑJàÔ“Û6ØOÞOÙã ÑÒMàÞׂÒ-àÒ-â¨ã ÞOßã ÑÒMÓ©Ù ÞˆéÒMã Þ%àÞׂÒVàÒ-â¨ãQÔOÛ)ÞˆâJØ%ã ÑJÒ?Õ¤ä©Ù¤ß,Ô“Ú-ÒàAÔ“ÓÕÒ-ÚVã Ò-Ü[Ú-Û)ÞˆÕ¤Ò-Ü»ÓÞˆÛCݩ؈ÞOâÕMÚ(ÔOÛ)Û)Ò(Ü¢—¦5©—ŸxŸc©—ª‚±A¯C ( 5¨¡ å8æQÔOÚ5ÑuÖ)âÜ©Ö)שÖ6Ü©äÔOÛÜJÖCÙÖ6ÞOâcÞ“ßã ÑÒ8Û6Þ¨ÞOÓcÖ)Õ Ú(ÔOÛ)Û)Ò(ÜxÔ%¡ež§±ˆŸžVª¢—å;ýrÚ-ÞˆÛ)Û)Ò-ÚVã Ö)ÞOâdÞOßÚVÞˆâJèÛ)Ö6âJÒ(Ô“ÙQÓÞ“Ù¤ãÒ(ÜGÕ¤Ò-ØOàÒ-â¨ã Õnã ÑÔãÙ Ò-ÓJÙ Ò-Õ¤ÒVâ‡ã ÕÁÕ¤ÑÔ“Ù Ò(ÜyéÞˆäâÜÔÙ¤ÝGéÒeã.ç1ÒVÒ-âGã´ç1ÞâÒ-ÚeãÙ ÒVßÃÒVÙ¤Ù Ò(Üdã ÞÁÔOÕ8ÔOâÆž¤«(±¨žVå ¥¦ÞOÙ¤ã Ö)ÞOâÕ8Þ“ß&ã ÑÒMÕ¤äJ٤ߗÔOÚ-ÒlÜJÞOàAÔOÖ)âÕ¤äJ٤ߗÔOÚ-Ò-Õ8Ö6ÕÑÒ¢àÞ‡ÜJÒVÛ§åÐ!Ù5ÔOâJÕ¤Ö)ã Ö)ÞOâÖ)âØÁÖ)Õ;Ö)âÜJÖ)Ú(Ô“ã Ò-ÜdÖ)ßã ÑJÒMÓJÙ ÞˆéÒ¨ ÕQàÞׂÒ-àÒ-â¨ãÞ“ßãã ÑÒ¢Û)Þ¨Ú(Ô“ÛÚ-Û)ÞˆÕ¤ÒVÕ@ã8ÓÞOÖ6â¨ãQã ÞnÚVÙ ÞOÕ¤Õ8ÔŠã¤Ù Ö)ààÖ)âØlÛ6Þ¨ÞOÓéÞˆäJâÜÔ٤ݨåÚ(Ô“äÕ¤Ò-ÕÞcã ÑÒÛ)ÒVßãŠÞ“ßQÔcÛ)Þ¨ÞˆÓÆÔÙ Ò%ÚVÞˆâÕ¤Ö6Ü©ÒVÙ Ò(Ü»Ú-ä©ã¤è´Ô(ç1Ô-Ý[çMÑJÖ6Û)ÒÁÓÖ)Ò-ÚVÒ-Õ?ã Þxã ÑÒãÖ)؈ѨãMÔ“Ù ÒŠÜJÒ-ÒVàÒ(ÜdÓÔÙ¤ã1ÞOß!ã ÑÒ?àÞ‡ÜJÒ-Û,åÿlÞOã ÒMã ÑÔã¢Ò(ÔOÚ5ÑAÕ¤äJ٤ߗÔOÚ-ÒŠã ÑÔ“ãÙÖ)ààÖ)âØcÛ)Þ¨ÞOÓ!åMû´ßã ÑJÒVÙ ÒÖ)ÕlÓÔ“Ù¤ãlÞOß;ÔdàÞ‡ÜJÒ-ÛÚ-ÞOâ‡ã5Ô“Ö)âÕlÔ“ãlÛ6Ò-ÔOÕ@ãnÞOâÒã¤ÙÖ)ÞˆâAÞOß&ã ÑÒMÕ¤ä©Ù¤ß,ÔOÚVÒléÒ-Ö)âJØ%ÚVäJã8Ô(ç1Ô-Ýcã ÑÒVâdã ÑÖ)Õ;Û)Þ¨ÞOÓ·Õ¤Ö)àÓJÛ)ÝÖ)Õ8âÞÓÞ“Ù¤ãÞ“Ù(åXòŠÞ(ç8Ò-ׂÒVÙ(ë ÖCãxÖ6ÕxÕ@ã Ö)Û6Û1ÔˆÜJ×OÔOâ¨ã5ԓ؈Ò-ÞOäÕxã ÞXêˆÒ-Ò-Óã ÑJÒÓÒVÙxÒ-âÜ©èÌÒVú&Ò-ÚVãÖ)ׂÒ[ã Þ+ÔGàÖ)âJÖ6àxäàuå}þŠÞˆÖ)âØGÕ¤ÞÕ Ô(ׂÒ-ÕÓ©Ù ÞOèâ‡äàxéÒVÙxÞOßàއܩÒ-Û)ÕÔOÚVãÞOäâÜ©Õ1ã ÑÒÁÜ©ÞˆàAÔOÖ)â[ÞOߦã ÑJÒÕ¤äJÙ¤ß,Ô“Ú-ÒˆåÕ¤äJÙ¤ÙÑÒ%àÞ‡ÜJÒVÛÖ6âyó¦Ö)ØJ妥“Ô[ÞOß8ÔAÕ¤Ö6àÓJÛ)ÒdÜJÖ)Õ¤Ú%çMÖ)ã ÑGÔAÑÞˆÛ)Òü8ÞˆâJÕ¤ÖÜ©ÒVÙãÖ)àÒdÔOâÜyàAÔ“Ö)â‡ã5Ô“Ö)âÕÑÖ)ØOÑäÓ&ÜJÔ“ã Ò%Ù5Ô“ã Ò-Õ-ådÐÞ»ã ÑÖ)ÕnÒVâÜë!Þˆä©ÙÚ-ÒVÕ¤Õ¤ÞOÙÁãÒ-àôÚ5ÑÒ-Ú5ê‡Õ¦ã ÑÒ¢Ó©Ù Þá©Ö)àÖCã.ÝÁÞ“ßã ÑÒ1Ó©Ù ÞˆéÒMÔOÕQÖCãQàޓ׈Ò-Õ¦ã ÑJÙ ÞOä؈ÑJÞˆä©ãÕ@Ý©Õ@ãÑJÙ ÞOäØOÑ}Ö)ã(åyÐ1ÑJÒã ÞˆÓ}Õ¤äJ٤ߗÔOÚ-ÒAçMÖ)Û)Û¢ÑÔ(ׂÒdã.ç8ÞXã¤Ù Ö6ààÖ)âJØ»Û)Þ¨ÞˆÓJÕÚ-äJã%ãÑÖ)âGÖCã ÕnÓÔ“Ù5Ô“àÒVã¤Ù Ö)ÚdÜJÞOàAÔOÖ)âyÔOÕÕ¤ÑJÞçMâyÖ)âó¦Ö)ة姥é!åÁÿlÞOã Ö)Ú-Òxã ÑÔ“ãçMÖCãÑJÒÒ-â‡×©ÖCÙ ÞOâàÒ-â¨ãã ÞXÒ(Ô“ÚeÑ}àއܩÒ-Û,å»ø+ÒäJÕ¤Ò·Ô»ùÙ Õ@ãxÞOÙ5ÜJÒeÙdÔ“ÓÓJÙ Þ(á©ÖCèãÖ6ÞOâ}ã ÞGã ÑÒ[Ú-Û)ÞˆÕ¤Ò-Õ@ãAÓÞOÖ)â‡ãcß—ÞˆäJâÜré¨Ý>ÔyàÒVã ÑJÞ‡Ü+Ù ÒVß—ÒVÙ¤Ù Ò-Ü>ã ÞGÔOÕàAÔãÑÒQÜJÖCÙ Ò-Úeã Ö6ÞOâÁÞO߇ã ÑÒ¦ã´ç1ÞMÛ)Þ¨ÞˆÓÕ!Ö)âÜJÖ)Ú(Ôã ÒQã ÑÔ“ãã ÑÒQÜÔÙ ê?Ù ÒV؈Ö)ÞˆâJÕÔ“Ù Òã ÞãÒVÙMÚ-Û)Þ¨Úeê¨çMÖ)Õ¤ÒÁÛ)Þ¨ÞˆÓXÚ-ä©ã ÕMÔ(ç1Ô-Ý[ã ÑJÒnÞˆä©ã ÒVÙ àÞˆÕ@ãéÒnÚ-ä©ãŠÔ-ç1Ô(ݨåQÐ1ÑJÒÞˆä©ãÖ)ÞOâGÓ©Ù ÞOèâJÞ‡ÜÔOÛàAÔ“ÓÓJÖ6âJØGì§Ð1ÑJÞˆàÓJÕ¤ÞˆâÆÒVã%ÔOÛ,å)ëQî(ïˆïˆ÷OÔ‚ñ5åAÐ1ÑÒxÕ¤Ö6àxäÛ6Ô“ãàÕMã ÑÖ)Õ?Ú-ÞˆàÓJäJã5Ô“ã Ö)ÞOâuÕ¤Ö)âÚ-ÒÁÖCãŠÖ)ÕlÔ؈Û)ÞˆéÔOÛ¦Ú-ÞOàÓä©ã5Ô“ã Ö)ÞˆâÚ-ÒVÕ¤ÕlÓÒVÙ¤ßÃÞOÙÒ-؈Ö)ÞOâ%çMÑJÖ6Û)Ò8ã ÑJÒ;Ö)âJâÒVÙÚ-ÞˆäJâ‡ã ÒeÙ Ú-Û)Þ¨Úeê¨çMÖ)Õ¤Ò1Û)Þ¨ÞˆÓ%Ú-äJã ÕÔ-ç¢Ô-Ýã ÑÒQÚ-Ò-â¨ã ÒVÙÙÒ-؈Ö)ÞOâAÙ ÒVÓJÙ Ò-Õ¤Ò-â¨ã Ö)âØnã ÑJÒ¢ÑÞOÛ6ÒOå Ð1ÑÒMÒ(ܩ؈Ò-Õ;Ö6âdó¦Ö6ة娥“éAÔÙ ÒMÖ6Û)Û)äÕ@ã¤Ù5Ôã Ò(ÜÙÑÔ“ãdÙ Ò1F¨äJÖ)Ù ÒVÕdã ÑÒ»Ó©Ù Þ¨Ú-Ò-Õ¤Õ¤Ö)âØÓÞç8ÒVÙAÞOßlã ÑÒ·ç1ÞOÙ ê‡Õ@ã5Ôã Ö6ÞOâ!åTøTÑÒ-âãÑJÒ?Ó©Ù ÞˆéÒMàÞׂÒ-Õ Ú-Û)ÞˆÕ¤ÒlÒ-âÞOäØOÑdã ÞÓÞOã Ò-â¨ã Ö6ÔOÛ)ÛCÝÚ-ÞOâ‡ã5Ô“ÚVã¢ÔnàއܩÒ-Û,ë‡ÔOâãÒVÙ âÔ“ã Ö)âØé©Ù Ö)؈Ѩã âÒ-Õ¤Õ1ã ÞxÖ)âÜJÖ)Ú(Ôã Ònã ÑÒnÓÔ“ã Ú5Ñ»Ö)âuó¦Ö)Ø©å©¥“Ôxã ÑÔãMÖ6ÕÖ)âuÔOÛCãÔOÚVÒ-â¨ã?ã ÞcÒ(ÔOÚ5ÑuÒ(ܩ؈҈åÔˆÜ1Ö)׈Ôã Ö6ÞOâXÓÔ“Úeê‚ÒeãlÖ)Õ?Õ¤Ò-â¨ãlã Þdã ÑJÒÑÔ“ÓJã Ö)ÚÁÓJÙ Þ¨Ú-Ò-Õ¤ÕlÖ)âÜJÖ)Ú(Ôã Ö)âØdã ÑÔãnÔÔ“ÚVãÑÒŠû´þ4ßÃÞOÙÚ-Û)ÞOÕ¤Ò-Õ@ã?ÓÞˆÖ)â¨ã¢Õ¤ÑÞOäÛ6Ü[éÒ?ã¤Ù5ÔOÚ5ê‚Ò(ÜåQÐ1ÑJÖ)Õ¢ÓÔOÚ5ê‚ÒVã1Ú-Þˆâ¨ã5ÔOÖ)âJÕ1ãã ÑJÒxàއܩÒ-ÛéÒ-Ö)âØ[ÔOÚVã Ö)×OÔ“ã Ò(ÜyÔOÛ)ÞOâØ[çMÖCã ÑÆã ÑÒ%Õ¤äJ٤ߗÔOÚ-Òdû´þÔ“âÜÆÓÔÙ5Ô“èÖ)ÚÛ)Þ¨Ú(Ôã Ö6ÞOâGÞ“ß ã ÑÒÁÚ-Û)ÞˆÕ¤Ò-Õ@ãnÓÞˆÖ)â¨ã(å?Ð1ÑÒ%ÓÔ“Ù5Ô“àÒVã¤Ù Ö)ÚÁ׈ԓÛ6äJÒã ÑÒ-âàÒVã¤ÙÑÒŠã¤Ù5Ô“Úeê‡Ö)âØdÔOÛ)؈ޓ٠Ö)ã ÑJàuåÕ¤Ò-Ò-ÜJÕ¢ãÑAã ÑÔ“ãMàއܩÒ-Û,ë‡Ö)Õ1Ù ÒVß—ÒVÙ¤Ù Ò-Ü·ã ÞdÔ“Õ1ã¤Ù5ÔOÚ5ê‡Ö6âJØJå8ü8Þˆâ¨ã¤Ù5ÔÙ¤Ýã Þxã¤Ù5ÔOÚVÖ6âJØJëçMÖCãÑJÒdÚ-Û)ÞˆÕ¤Ò-Õ@ãxÓÞˆÖ)â¨ãÁß—Þ“Ù%ã¤Ù5Ô“Úeê‡Ö)âØ»àcäÕ@ãÁéÒxã ÑÒ؈Û)ÞOéÔOÛ;Ú-Û)ÞˆÕ¤Ò-Õ@ãxÓÞˆÖ)â¨ã(åãÒ-Õ?ÓJÙ ÞOÓÒVÙ?Ú-Þˆâ¨ã5Ô“ÚVãlÜJÒVã Ò-Úeã Ö6ÞOâué¨Ý·ÓÒVÙ àÖCã¤ã Ö6âJØcã ÑÒÚVÛ6ÞOÕ¤Ò-Õ@ãÐ1ÑJÖ6ÕMÒ-âJÕ¤äJÙÞ @äàÓrÔOÚeÙ ÞˆÕ¤ÕÚ-ÞOâÚ(Ô(ׇÖCã Ö6ÒVÕÔOâÜrã ÞÆÚ-Û)Ö)àcérÚ-ÞOâ©×ˆÒVá+Ù Ò-ØOÖ6ÞOâÕ-åÓÞOÖ6â¨ãxãì§Ô‚ñÒVÕ?ã ÑJÒxÜ©ÖCú&ÒVÙ Ò-â¨ãŠÚVÛ6ÞOÕ¤Ò-Õ@ãÓÞOÖ6â¨ã?Ù ÒVèóÞ“ÙŠÒeáÔ“àÓÛ)Òˆë&ó¦Ö)Ø©å.¢AÜJÒ-àÞOâÕ@ã¤Ù5Ô“ãÒ-àÒVâ‡ã Õ-å»þnäJÙ Ö)âJØ»ã¤Ù5ÔOÚ-Ö)âJØJë ã ÑÒAÓJÙ ÞOéÒAàޓ׈Ò-ÕßÃÙ ÞˆàoÓÞˆÕ¤ÖCã Ö)ÞˆâF¨äÖCÙ4&# 4+ 8 8 % $1$© %-*$78Õ¤ä©Ù¤ß,Ô“Ú-ÒÙÒ-Õ@ã¤Ù Ö)ÚVã Ò(ÜrßÃÙ Þˆà²àޓ׈Ò-àÒ-â¨ã%Ö6ârã ÑÔãAÜJÖCÙ Ò-Úeã Ö6ÞOâ!å>û´ßÔ“âÜ}Õ¤ÑÞOäÛ6Ü>éÒ·ÙÑJÒyÕ Ô“àÒ+Ô“Û6ØOÞOÙ ÖCã Ñà ç¢Ô“ÕuäÕ¤Ò(ÜTßÃÞOÙuã¤Ù5Ô“Úeê‡Ö)âØ©ëÔrÕ¤Ö6àÖ)Û6Ô“ÙuÕ¤Ú-Ò-âÔÙ Ö)ÞãÞ|ßÃÒ-Ò-ÛÁã ÑÒ+Ô“ÚVã äÔ“Û%ÜJÒ-Õ¤Ö)؈âJÒ(Ü4àއܩÒ-ÛÁÖ)âÕ@ã Ò(ÔOÜ3Þ“ßÕ¤ÞˆàÒGÕ¤Ò-Ú-ÞˆâÜÔÙ¤ÝãÒ-ÓJÙ ÒVÕ¤Ò-â¨ã5Ô“ã Ö)Þˆâåû.âGÔˆÜJÜJÖCã Ö)ÞˆâGã Þ»ÔOâyÒ-âÑÔOâJÚ-Ò(Üyã¤Ù5Ô“Ú-Ö)âØ[ÒVá©ÓÒVÙ Ö)Ò-âÚ-ÒOëÙÞ»ÔOâyÒ(ÜJØOÒˆådû´âyã ÑJÖ6ÕÚ-ÔOÕ¤ÒˆëÑÞ(ç1Ò-׈ÒVÙ(ë!ã ÑJÒdÚ-Û)ÞˆÕ¤ÒVÕ@ãxÓÞOÖ)â‡ãnÕ¤ÑÞˆäJÛ6ÜGâÞ“ããÑÒÁÓ©Ù ÞˆéÒÁç1ÒVÙ Ò%ã Þ·ÚVÞˆâ¨ã Ö)â©äJÒ%ÔOÛ)ÞˆâJØ·ã ÑJÒxÚ-ä©Ù¤èéÒÁéÞˆäJâÜ&åŠû´âuß,ÔOÚeã(ë&Ö)ßQãÑJÒ»Ô“ÚVã äÔ“Û¢àÞ‡ÜJÒ-Û8ÔOÛ)Õ¤ÞÔ“Û)Û6Þ(çMÕdã ÑJһܩÒ-Õ¤Ö)؈âÒeÙcã ÞyàÞ‡ÜJÖCßÃÝGã ÑÒäÕ¤Ö)âØXãÑJÞˆäJãnÑÔ(ׇÖ6âJØ·ã Þ·ç¢Ô“Ö)ãßÃÞOÙã ÑJÒxÑÔOÓJã Ö)ÚdÕ@Ý©Õ@ã Ò-à{ã Þ[ÚVÞˆâ‡×‚ÒVÙ¤ãàÞ‡ÜJÒ-Û¦çMÖCãÒ-â¨ã·ÓÔã Ñ#ÔOâÜTÖ)â‡ã ÒeÙ Õ¤Ò-ÚVã·ã ÑÒXàÞ‡ÜJÒVÛ§ëMã ÑJÒXÚ-Þˆâ¨ã5ÔOÚeã[ç8ÞˆäÛ6ÜTâJÞOã[éÒÙÒ-ÚVã Ò(ÜÆÕ¤Ö)âÚ-Òxã ÑÒdÓÒ-âJÒVã¤Ù5Ô“ã Ö)ÞOâGç1ÞOäÛ6ÜGâJÞOãÁÞ¨Ú-ÚVäJÙ%Ô“ãã ÑJÒcÛ)Þ¨Ú(Ôã Ö6ÞOâÜ©ÒVãÑÒ[àÞ‡ÜJÒ-Û¢Ö)â|Ôyã Ö)àÒVèÌÚ-ÞˆâJÕ¤äàÖ)âØÆÓ©Ù Ò-ÓJÙ Þ¨Ú-ÒVÕ¤Õ¤Ö6âJØ+Õ@ã Ò-Óå`Ð1ÑJÒXÜJÖGdèãÑÖ)ÕAÔOÓJÓJÙ Þ‚Ô“ÚeÑ|Û)Ö6ÒVÕ·Ö)â>ã ÑÒuÒ-×OÔOÛ)äÔ“ã Ö)Þˆâ|ÞOßnã ÑJÒuàÞ‡ÜJÒ-ÛÚ-äÛCãAÓÔ“Ù¤ãAÞOߊãޓߦã ÑÒnéÞOäâÜ·ã¤Ù5ÔOÚ5ê‚Ò(Ü·ÓÞˆÖ)â¨ã(åA91

§ ì¦ ¤¦¨! "§¦ñ5ëOÚ(ÔOÛ)Ú-äJÛÔã Ò(ÜcäJÕ¤Ö6âJØlÙ ÒVùâÒ-àÒVâ‡ã8ìÌü8ÞˆÑJÒ-âcÒVãQÔ“Û,å6ëJî(ï¨ ¡§¨ñçMÑÒeÙ Ò¦ ! " ¦ ñ8ÔÙ Ònã ÑÒÓÔÙ5ÔOàÒVã¤Ù Ö)ÚnÚVÞ¨ÞOÙ5ÜJÖ)âÔ“ã Ò-ÕMß—Þ“Ùlã ÑJÒnÓÞˆÖ)â¨ã?ÞOßÚ-ÞOâ¨ã5ÔOÚVã(åì¦ ¨© ¨¢ ! ¨"© ¨¤ " å{Ð1ÑÒ+Ú-ÔOÛ)Ú-äÛ6Ôã Ö6ÞOâ Þ“ß¤ì¦ ¤¦¨! "§¦ñ;é¨ÝÙ ÒVùâJÒ-àÒ-â¨ã8Ù ÒVÕ¤äÛCã ÕMÖ6â·ã ÑÒlâÒVç3ê‡âÞ“ã¢×‚Ò-ÚVã Þ“Ù Õ>¤ @ ÔOâÜ$ ì¦ ¦ ! " ¦ ñ5ånóä©Ù¤ã ÑÒVÙ àÞ“Ù Òˆëã ÑJÒ ¦ ÚVÞˆÛ)äàâÆÔ“âÜ»ã ÑÒ£ ¦ Ù Þ(ç 0 0 +.- 68 *54 9=£6 * @ ÔOâÜ[ÔÁâÒeç#Ú-Þˆâ¨ã¤Ù ÞOÛàÒ-Õ¤Ñ > Ð1ÑÒŠã¤Ù5Ô“Ú-Ö)âØcÔOÛ)؈ޓ٠Ö)ã ÑJàÚ(Ô“ââÞ“ãMéÒŠäJÕ¤Ò(Ü·ÒeáJÚVÛ6äJÕ¤Ö)ׂÒ-ÛCÝß—Þ“Ù¢ã¤Ù5ÔOÚ5ê‡è2 *,4 0A@ çMÑÒeÙ Ò2 *4 0 ÑÒQÓJÙ ÞOéÒ8Ö)ÕÖ)â%Ú-Þˆâ¨ã5Ô“ÚVãÔ“âÜnã ÑÒVÙ Òeß—ÞOÙ ÒQã ÑÒQâÒVá‡ãÖ)âØ?Õ¤Ö6âJÚ-Ò8ÖCã¦ÔOÕ¤Õ¤äàÒVÕ!ãÒ-Û6Ôã ÒŠã ÞcÔÁàÞׂÒ-àÒ-â¨ãQéÞˆäJâÜdã ÞÜJÒ-Õ¤ÖCÙ5Ô“éÛ)ÒnÚ-Û)ÞˆÕ¤ÒVÕ@ãMÓÞˆÖ)â¨ã¢Õ¤ÑÞOäÛ6ÜÚ-ÞOÙ¤ÙÖ)âxã ÑJÒ1Ú-ÞOâ‡ã¤Ù ÞOÛJàÒVÕ¤Ñ%ßÃÞOÙ àÇÚ-Þˆâ¨ã¤Ù ÞOÛJÓÞOÛ)Ý©ØOÞˆâÕßÃÞOÙQÖ)Õ¤Þ¨Ú-äJ٠ׂÒVÕ ã ÑÔ“ã ÓÔOÕ¤Õ2 * 4 0 ÑÒ%Õ¤äJ٤ߗÔOÚ-ÒdÞ“ß;ã ÑJÒxàއܩÒ-Û,ånû´âÆã ÑJÒ%ã¤Ù5Ô“Ú-Ö)âØ[ÔOÛ)؈ޓ٠ÖCã ÑàuëÖCãÖ)ÕÔOÓJÓJÙ ÞOèãÖ6Ô“ã ÒnßÃÞOÙMã ÑÒÚVÛ6ÞOÕ¤Ò-Õ@ã?ÓÞˆÖ)â¨ãMã ÞdÑÞOÛÜ·ÖCã Õ?ÓÞOÕ¤ÖCã Ö6ÞOâXÕ¤Ö)âÚ-ÒÖCã ÕMÓä©Ù ÓÞˆÕ¤ÒÓJÙã Ñ©Ù ÞˆäØOÑ[ã ÑJÒÓÞˆÖ)â¨ã å;Ð1ÑÖ)Õ?ÔOÛ)Û)Þ(çMÕlã ÑÒÚ-ÔOÛ)Ú-äÛ6Ôã Ö6ÞOâXÞOßã ÑJÒnã.ç1Þã5Ô“â؈ÒVâ‡ãA׈Ò-ÚVã ÞOÙ Õdã ÞéÒuÓÒV٤ߗޓ٠àÒ(Ü|äJÕ¤Ö)âØã ÑJÒuàÞOÙ ÒuÕ¤Ö)àÓJÛ6Ò»Ú-äJ٠ׂÒÖ)Þˆâå Ð1ÑÒMÖ)Õ¤Þ¨Ú-äJ٠׈ҊܩÒVùâJÒ(ÜdÔ“ã " ¦ Ö)âdã ÑJÒ ¦ Ù Þç`ÞOß&ã ÑÒ¢ÚVÞˆâ¨ã¤Ù ÞˆÛÒ1F¨äÔ“ãàÒ-դѷÖ)ÕlØOÖ)ׂÒ-â»é¨ÝÞÙ Ò-Õ@ã¤Ù Ö)ÚVã8ã ÑJÒlÓ©Ù ÞˆéÒ¡ Õ;àޓ׈Ò-àÒ-â¨ã Ö)âÚ-ÒVÙ¤ã5ÔOÖ)â[ÜJÖCÙ Ò-ÚVã Ö)ÞOâÕ-å þnä©Ù Ö6âJØÖ)Õ;ãÑÒVÙ ÒMÕ¤ÑÞOäÛ6ÜdéÒ?âÞnÙ Ò-Õ@ã¤Ù Ö)ÚVã Ö)ÞˆâJÕ1Þˆâdã ÑJÒMÓJÙ ÞˆéÒ¨ Õã¤Ù5ÔOÚ5ê‡Ö6âJØJë‡ÑJÞç8Ò-ׂÒVÙ(ëˆãÑÒnÚ-Û)ÞOÕ¤Ò-Õ@ãŠÓÞOÖ)â‡ãMçMÑJÖ6Ú5ÑuÕ¤ÑÔˆÜJÞ(çMÕMÖCã(åàÞׂÒ-àÒ-â¨ã1Þ“ÙMãÒ-â¨ã ÛCݨë&ÖCãÁÖ)ÕnâÞ“ãnß—Ò(Ô“Õ¤Ö)éÛ)Òdã ޻ܩÒVã ÒVÙ àÖ)âÒÁã ÑÒx؈Û)ÞˆéÔOÛ Ú-Û)ÞˆÕ¤ÒVÕ@ãü8äJÙ¤Ù2 *ì¦ ñ%$ &)(*,+.-ÞnÔ¢ã¤Ù Ö)ààÒ(Ünÿ£ ¡¢£¤1öàއܩÒ-Û‡Ô“ã ÑÔOÓ©ã Ö6ÚQÙ5Ô“ã Ò-Õ-åÐ1ÑÒVÙ ÒVßÃÞOÙ ÒˆëOç1ÒÓÞˆÖ)â¨ã¦ãÖ6ÜcÔ“ÓÓ©Ù Þ‚ÔOÚ5ÑcçMÑJÒVÙ Ò¢ã ÑÒ1ã¤Ù5ÔOÚVÖ6âJØÔOÛ)ØOÞOÙ ÖCã Ñà Ö)Õ8Ô“ä؈àÒVâ‡ã Ò-ÜäÕ¤ÒMÔnѨݩéJÙ& (6 * @ ÔÙ ÒÖ)އܩÖ6ÚAÙ ÒVèÌÕ¤Ò-Ò(Ü©Ö)âØXçMÖCã Ñ+ã ÑÒA؈Û)ÞˆéÔOÛ1ÚVÛ6ÞOÕ¤Ò-Õ@ãÓÞOÖ6â¨ã(åÆÐ1ÑJÒAÕ¤Ö6àdèé¨ÝÓÒVÙÖ)ÞOầ ÓJÙ Þ¨ÚVÒ-Õ¤ÕAÚ(ÔOÛ)Ú-äJÛÔã Ò-Õ[ÔOâ|ÔOÓÓ©Ù Þ(áJÖ)àAÔã һ؈Û)ÞOéÔOÛlÚ-Û)ÞˆÕ¤Ò-Õ@ã·ÓÞOÖ6â¨ãäÛ6Ô“ãçMÑJÒVÙ Ò¢ã ÑÒ > ã ÑJÒcÔ“Õ¤Õ¤Þ¨Ú-Ö6Ô“ã Ò(ÜGç8Ò-Ö)؈Ѩã Õ-ëÔOâÜXã ÑÒ > 8 *54 9 < @ Ô“Ù Òxã ÑJÒcéÔOÕ¤Ö)Õnß—äJâÚVã Ö)ÞˆâJÕ2 * @ ß—Þ“Ù àôã ÑÒMÒVá‡ã¤Ù5ÔOÚVã Ò-ÜdÚ-Þˆâ¨ã¤Ù ÞˆÛJÓÞˆÛCݩ؈ÞOâ!ë¨ã ÑÒ>£éÔOÕ¤Ò-Ü[Þˆâ»Ô%ã Ö)àÒVèÌÚVÙ ÖCã Ö)Ú(ÔOÛ&àÒVã ÑJÞ‡ÜAã ÑÔ“ãMÕ¤ÓÒ-âÜJÕäÕ¤Ö)âØcÔOâ»ÔOÛ)؈ޓ٠Ö)ã ÑJàÖ)àÒÞOâuÞˆé ¤Ò-Úeã ÕŠÞˆä©ã Õ¤Ö6ÜJÒnã ÑÒnÙ ÒV؈Ö)ÞˆâÆÞOß Ö)â¨ã ÒVÙ Ò-Õ@ã%즥‚ÞOÑâJÕ¤ÞˆâÆÔOâÜÛ)Ò-Õ¤Õ?ãÖ)Ú·Ó©Ù Þ¨Ú-Ò-Õ¤ÕcÔOÚ-Ú-ÒVÓJã ÕÁã ÑÖ)ÕcÓÞOÖ6â¨ãcÔOâÜäÕ¤Ò-Õü8ÞˆÑJÒ-â!ë¢î-ïˆï‚÷ˆñ5åGÐ1ÑÒ·ÑÔOÓ©ãÑJÒ-âôÚ(Ô“Û6ÚVäÛ6Ô“ã Ò>ã ÑÒ}ã5ÔOâJ؈Ò-â¨ãGׂÒ-ÚVã Þ“Ù+é¨Ý ÒV׈ԓÛ6äÔ“ã Ö)âØ#ã ÑJÒøÒrãÖ)׈Ôã Ö6׈ÒlÞ“ßã ÑÒlÖ)Õ¤Þ¨Ú-äJ٠ׂÒnÔ“ã ¦ å lÕ¤Ö6âJØ%ã ÑJÒF¨äÞ“ã Ö)Ò-â¨ã1Ù äÛ)Òlã ÞÁß—Þ“Ù àÜ©ÒVÙÞÙ ÒVèÌÕ¤Ò-Ò(Üuã ÑÒã¤Ù5Ô“Ú-Ö)âØ[ÔOÛ)؈ޓ٠Ö)ã ÑJàuånÐ1ÑJÖ)ÕŠÓÒVÙ Ö)Þ‡ÜJÖ)Ú%Ù ÒVèÌÕ¤Ò-Ò(ÜJÖ)âJØ[Ô“ÛCèÖCãŠãÑJÒlã¤Ù5ÔOÚVÖ6âJØdÔ“Û6ØOÞOÙ ÖCã Ñà ã Þxß—Þ“Ù à ÔÁ؈ިއÜuÔ“ÓÓJÙ Þ(á©Ö)àAÔ“ã Ö)Þˆâã Þ%ã ÑÒÛ)ÞçMÕ¢ãã ÑJÒnׂÒ-Û)Þ¨Ú-ÖCã.Ý[Ú-ä©Ù ׂҊߗޓÙlæ Få!쥂ñ;Ý©Ö6ÒVÛÜ©Õ-ëÒ-Õ-å øÒÑÔ(ׂҊßÃÞˆäâÜ[ã ÑÖ)Õ¢ã ÞéÒ؈Û)ÞˆéÔOÛ¦ÚVÛ6ÞOÕ¤Ò-Õ@ãlÓÞˆÖ)â¨ãléÒeã.ç1ÒVÒ-âuäÓ&ÜJÔ“ãÖ6׈ÒÕ¤Ö6âJÚ-Òã ÑÒdÓ©Ù ÞˆéÒdã´ÝJÓJÖ)Ú(ÔOÛ)ÛCÝ+Ü©Þ¨Ò-Õ%âÞOã%àÞׂÒd׈ÒVÙ¤ÝÆß,Ô“ÙׂÒVÙ¤ÝyÒVú&Ò-Úeã%'&ì¦ ñ"!2 **,4 9

Þˆâ%ã ÑÒ8âJÒ-Ö)؈чéÞOÙ Ö)âJØnÕ¤äJ٤ߗÔOÚ-ÒˆåÐ!Ù5ÔOÚ-Ö)âJØÔOÛ)ÞˆâJØÔ?ã¤Ù Ö)à¬éÞOäâÜÔ“Ù¤ÝÓÞˆÖ)â¨ãÑÒ § ¨ |àÞ‡ÜJäÛ)ÒXçMÖCã Ñ#ã ÑÒ ¨¨¨ àއܩäÛ)ÒÖ)Õ»ÑÔ“âÜ©Û6Ò-ÜTé‡Ý|ã! "¢?&# 4(!)% 5 ( $* 4&8 -*$1 $ ¢A3 ( "1 # !! %#(#:& "!!#" % $1&#!;(#"¡ -C4&# 4(!)5;*534! 3 ©)!*$33¦ 5£(;:+ 8 $ %"(!/5¢ /Á%V$¥¨§&¥¨ û´â[Þ“Ù5ÜJÒVÙ¢ã ÞdÕ¤àÞ¨Þ“ã ÑÛCÝã¤Ù5ÔOâJÕ¤ÖCã Ö6ÞOâ»ßÃÙ ÞOà§6I§ÞyÔOâÞ“ã ÑÒVÙxÖCãcÖ)ÕxâÒ-Ú-ÒVÕ¤Õ Ô“Ù¤Ý+ã ÞÆÚ(Ô“Û6ÚVäÛ6Ô“ã Ò[ÔOârÔ“Ú-Ú-äJÙ5Ôã ÒÞOâÒAÕ¤äJ٤ߗÔOÚ-ÒAã! !>înçMÑÒeÙ Ò'&iÖ)ÕMã ÑJÒnâ‡äàxéÒVÙ¢ÞOߦդÒ-؈àÒ-â¨ã Õ¢Ö)â»ã ÑJÒnÒ(ÜJØOÒÁÔOâÜ &ÑJÒlÖ)âÜJÒVácÞOßã ÑÒ?Õ¤Ò-؈àÒ-â¨ã;Ö)â‡ã ÒeÙ Õ¤Ò-ÚVã Ò(Ü&åQÐ1ÑÒ?ÒVáJÔOÚVã8ÓÞˆÖ)â¨ã1ÞOßã¤Ù5ÔOâ©èÖ)Õ;ã$C'()£!& P8¡Ò-Ò¢éÔOÕ¤Ö)Ú1ã¤Ù5ÔOâJÕ¤ÖCã Ö6ÞOâÕQÚ(ÔOâxÞ¨Ú-Ú-ä©Ù8Ü©äJÙ Ö)âØŠÔlã¤Ù5ÔOÚVÒˆå Ð8ç1Þlß—ÞOÙ àÕÐ1ÑJÙEnd-effector position§6IÒ-Õ¤äÛCã¢çMÑJÒ-â·ã ÑJÒŠÓ©Ù ÞˆéÒ¡ Õ¢àÞׂÒ-àÒ-â¨ã;Ú(ÔOäJÕ¤Ò-Õ1ã ÑÒlÛ)Þ¨Ú(ÔOÛÚ-Û)ÞˆÕ¤Ò-Õ@ã?ÓÞOÖ6â¨ãÙÞXÑÖCãcÔ»ã¤Ù Ö6àoéÞˆäJâÜJԓ٤ݨåuû´âã ÑÒùJÙ Õ@ã%ßÃÞOÙ àoÞOßMã¤Ù5ÔOâJÕ¤ÖCã Ö6ÞOâÖ)âØ©ë ã ÑÒãÖ)ÞOâ9ÔOÚVÙ ÞOÕ¤Õ·ã ÑJÒ»ã¤Ù Ö)àvÔOâÜ>Þˆâ¨ã ÞÛ)Þ¨Ú(ÔOÛnÚVÛ6ÞOÕ¤Ò-Õ@ã[ÓÞˆÖ)â¨ãAÕ¤ÑÞˆäJÛ6Ü|ã¤Ù5ÔOâJÕ¤Ö)ãÕÁàÞˆÕ@ã%Þ“ßÃã Ò-â+Ö)âÔ“Ù Ò(Ô“ÕÔOâ}ÔOܤÔOÚ-Ò-â¨ã%Õ¤äJ٤ߗÔOÚ-ÒÆì§ó¦Ö)ØJå3‚Ô‚ñ5åuÐ1ÑJÖ)Õ%Þ¨ÚVÚ-äJÙOn trimyesslide3dnonext_uv check_crossnoEvaluateyesfind_adjacentäÚVã Ò-Ü|é¨Ý>ÓÔOÕ@ã Ö)âJØ+Õ¤äJ٤ߗÔOÚ-Ò-ÕAã ÞˆØOÒVã ÑÒeÙ»Ô“Û)ÞˆâØrÔ“âÞOß%ÔyàÞ‡ÜJÒ-Û?Ú-ÞˆâJÕ@ã¤ÙÔOÚVÒ-â¨ãcÒ(ܩ؈҈å»Ð1ÑÒÕ¤ÒVÚ-ÞˆâÜÓÞˆÕ¤Õ¤Ö)éÖ)Û)ÖCã.ÝÖ6ÕÁã ÑÔãÁã ÑÒdÚ-Û)ÞOÕ¤Ò-Õ@ãdÓÞOÖ6â¨ãÔˆÜ1Ò-àAÔ“Ö)âxÞOâ%ã ÑJÒ8ã¤Ù Ö)à Ò(ܩ؈҈å Ð1ÑJÖ)ÕÖ6Õ¦ã ÑJÒ1Ú(Ô“Õ¤Ò1Ö)âcÚ-ÞOâÚ(Ô(ׂÒ8Ô“Ù Ò(Ô“ÕÕ¤ÑÞOäÛ6Ü%ÙÒlã ÑÒŠÓ©Ù ÞˆéÒnÚ-ÔOâ[ã¤Ù5Ô“Ú-ÒÔ“Û6ÞOâØ%ã ÑÒnÖ)â¨ã ÒVÙ Õ¤Ò-ÚVã Ö)ÞOâ»ÞOß!ã.ç8ÞcÕ¤äJ٤ߗÔOÚ-ÒVÕçMÑÒVÙì§ó¦Ö)ØJå.ˆéñ5ånÐ1ÑJÒ%ùâÔ“Ûß—Þ“Ù àÖ)Õlã ÑJÒxÕ¤ÓÒ-ÚVÖÔ“ÛÚ(ÔOÕ¤ÒxÞOß ã¤Ù5ÔOâJÕ¤ÖCã Ö6ÞOâÖ)âØ·ÞOúÑÒMàÞ‡ÜJÒVÛ§åø#ÑÒ-âxã ÑÒ¢ÓÒ-âJÒVã¤Ù5Ô“ã Ö)Þˆâ·Ü©Ò-ÓJã ÑxéÒ-Ú-ÞˆàÒVÕ;âJÒ-Ø‚Ôã Ö6׈҈ë¨ã¤Ù5ÔOÚVèãÒ-Õ¤äJàÒ-Õ-ë‚çMÑÖ)Ú5ÑAÖ6Õ;ÒVú&Ò-ÚVã Ö)ׂÒVÛ)Ýxã ÑÒMÖ)â‡×‚ÒVÙ Õ¤Ò?ÞOßÖ)âØÁÒ-âÜ©Õ8Ô“âÜcã¤Ù5Ô“Úeê‡Ö)âØÙreleasenoEvaluateyesnext_uvyeson_surfacenoslide3dã ÑÒnÚVÞˆâ¨ã5ÔOÚVã?Ó©Ù ÞˆéÛ)Ò-àuåspecialnormalEvaluatespecialnormalÑ>ÔyØ“Ù Ö6Üå}æQÔOÚ5ÑrÚ-Ò-Û)Û¢Ö)â}ã ÑÒ·ØOÙ Ö6Ü>ÚVÞˆâ¨ã5ÔOÖ)âÕ%ã ÑÒAã¤Ù Ö)à Õ¤Ò-ØOàÒ-â¨ã ÕçMÖCãÑÔ“ã%Û)Ö6ÒçMÖCã ÑÖ)â}Þ“ÙcÖ)â¨ã ÒVÙ Õ¤Ò-ÚVãxÖCã(åXû@Ü©Ò(ÔOÛ)ÛCݨëÒ(Ô“ÚeÑ}ÚVÒ-Û)Û8ç1ÞOäÛ6Ü+Ú-Þˆâ¨ã5Ô“Ö6âãÛCÝÞOâÒ»Õ¤ÒV؈àÒ-â¨ãÔ“âÜrÒ(ÔOÚ5Ñ>Õ¤Ò-؈àÒ-â¨ãdç8ÞˆäJÛÜréÒ»ÚVÞˆâ¨ã5ÔOÖ)âÒ-Ü>Ö)â>ÒVáJÔOÚVãÖ)Ú-Ò·ã ÑJÖ)ÕcÖ)ÕcâJÞOãxâÒ-Ú-ÒVÕ¤Õ Ô“Ù¤ÝrÔ“âÜ+ç1ÞOäÛ6ÜÙ Ò-Õ¤äÛCãcÖ6âÞOâÒAÚ-Ò-Û)Û,åû´â}ÓJÙ5Ô“ÚVãÒ-Ó©Ù Þ¨Ú-Ò-Õ¤Õ¤Ö)âØ+ÞׂÒeÙ ÑÒ(ÔOÜå`øÒ»Û)Þ¨Ú(Ôã Òuã ÑÒuØ“Ù ÖÜ>Õ¤ÞGã ÑÔ“ãAÖCã ÕÑJÒ(Ô(ׇÝ+ÓJÙÑ·ã ÑJÒŠéÞˆäJâÜ©Ö6âJØÁéÞáAÞ“ßÔ“Û6Ûã¤Ù Ö)à Õ¤ÒV؈àÒ-â¨ã Õ-åéÞOäâÜJÔ“Ù¤ÝÚVÞˆÖ)âÚ-Ö6Ü©Ò-Õ1çMÖCãì,éñì§Ô¨ñ *¡ 6(£¢E*( #3¦ ¦ $1 ! %(#)!*$33 ( 4* 8;8 5;5"?( #" $ %$ ( #$!:&#! 34-*(#)5¤¢E(1 %3¦ ¦ $ ! 5 $ %$©4&%; %6!*35)0¦ $1 ©$#-C ¢ $ 34-*(#)5530ÑÒVÙ(ë©ç1Ò%Ú-ÞOâÕ@ã¤Ù äÚeãlã ÑJÒ%Ø“Ù Ö6Ü»ã ÞÑÔ(ׂÒß—ÞOäJÙ?ã Ö6àÒVÕŠÔ“ÕlàAÔ“â‡Ý[Ú-Ò-Û)Û)Õóä©Ù¤ãÕ¢çMÖ)ã Ñuã ÑJÒâ‡äàcéÒeÙMÞOß Ù Þ(çMÕŠÔ“âÜuÚ-ÞOÛ6äJàâÕMéÒ-Ö)âØÒF‡äÔOÛ,åÔ“ÕlÕ¤Ò-ØOàÒ-â¨ãÖ)Ú-Òˆë©ã ÑÖ)ÕMÖ)Õ?ÑJÒ-äJÙ Ö)Õ@ã Ö)ÚÑÔ“ÕMÓJÙ ÞׂÒ-â·ã ÞdéÒnÒVú&Ò-Úeã Ö6׈҈åû´â»ÓJÙ5Ô“ÚVãÞ ¥§¦£¨£¥$©¥£ Ù Ò-Õ¤äJÛ)ã ÕQÖ)âÞˆâJÛ)ÝxÚ5ÑÒ-Ú5ê‡Ö6âJØnã ÑÞOÕ¤Ò?Õ¤ÒVØOèæQÔOÚ5ÑÚ(ÔOÛ)Û©ãÕ¢ÛCÝ©Ö)âØçMÖCã ÑÖ)âuã ÑÒÁÚ-Ò-Û)Û)Õlã ÑJÒàÞׂÒ-àÒ-â¨ãMׂÒ-Úeã ÞOÙlÖ6â¨ã ÒVÙ Õ¤ÒVÚVã Õ-åMû.âàÒ-â¨ãÖ)Þˆâëˆç8Ò?äJÕ¤ÒMÔnØOÙ Ö6Üxç1ÔOÛ)ê‡Ö6âJØÁÔOÛ)؈ޓ٠ÖCã Ñà Ö6âcÞOÙ5ÜJÒeÙQã ÞÚ5ÑÒVÚeêã ÑÒVÕ¤ÒÔOÜÜJÖCãÒdÔ“Ù ÒcßÃÞˆäJÙ%Ú-Þ“Ù ÒdàÞ‡ÜJäJÛ)Ò-ÕŠã Þ[ã ÑJÒxã¤Ù5Ô“Ú-Ö)âØXÔ“Û6ØOÞOÙ ÖCã Ñàiã ÑÔ“ãÐ1ÑÒVÙàÖCãcã ÑJÒ[Ó©Ù ÞˆÓÒVÙAÜ©ÒVã Ò-ÚVã Ö)ÞOầ Ô“âÜrÑÔOâÜJÛ)Ö)âØyÞOß?ã¤Ù5ÔOâJÕ¤Ö)ã Ö)ÞOâÕ-å|Ð1ÑÒÓÒVÙÑJÒ%Þ“Ù5ÜJÒVÙlã ÑÒÁàÞׂÒ-àÒ-â¨ãMׂÒVÚVã ÞOÙ?ã¤Ù5Ô(ׂÒVÙ Õ¤Ò-Õ?ã ÑJÙ ÞˆäJ؈Ñuã ÑJÒ-àuåÚ-ÒVÛ6Û)ÕlÖ6âXãÒeÙ Õ¤Ò-ÚVã Ö)Þˆâ+Ú5ÑÒVÚeê‡ÕÚ-ÞOâÚ-Û)äÜ©ÒÔ“ãã ÑJÒcùÙ Õ@ãÁ×OÔOÛ)Ö6ÜGÖ)â¨ã ÒVÙ Õ¤Ò-Úeã Ö6ÞOâ!ëÐ1ÑJÒdÖ)â‡ãàÞ‡ÜJäJÛ)ÒxÜ©ÒVã Ò-ÚVã Õ?çMÑÒ-âÆÔàÞׂÒ-àÒVâ‡ã?Ô“Û)ÞˆâØ[ÔdÕ¤äJ٤ߗÔOÚ-Ò¥§¦£¨£¥©¥£ÒeÙ Õ¤Ò-ÚVã ÕÁÔ·ã¤Ù Ö)ààÖ)âØAÛ)Þ¨ÞˆÓå âJÚ-ÒcÕ¤äJÚeÑGÔOâGÖ6â¨ã ÒVÙ Õ¤ÒVÚVã Ö)ÞˆâÖ)ÕnßÃÞˆäâÜÖ)â‡ãÑÒ§ £ ¥¨ [àÞ‡ÜJäJÛ6ÒÁÜ©ÒVã ÒVÙ àÖ)âÒVÕMã ÑJÒÁÒVáJÔOÚVãlÚ-ÞOÙ¤Ù ÒVÕ¤ÓÞˆâÜ©Ö)âØãÑÒeÙnÚ-ä©ã¤ã Ö6âJØAÜJÞ(çMâÆÞˆâÆã ÑÒÁâ‡äàxéÒVÙ?ÞOß;Ö6â¨ã ÒVÙ Õ¤ÒVÚVã Ö)ÞˆâyÚeÑJÒ-Úeê‡ÕlÓÒVÙ¤èßÃäJÙ¤ãàÒ(Ü&å ßÃÞOÙÞÁÜJÒVã ÒeÙ àÖ6âJÒ1çMÑÒ-âdã ÑJÒ¢Ò(ÜJØOÒ?ã¤Ù5Ô“Ú-Ö)âØÕ¤ÑJÞˆäÛ6Ücã ÒVÙ àÖ)âÔ“ã ÒéÒ-Ö)âØäJÕ¤Ò(ÜcãàAÔ“ÛÕ¤ä©Ù¤ß,ÔOÚVÒ?ã¤Ù5Ô“Ú-Ö)âØÁÕ¤ÑÞOäÛ6ÜcÙ Ò-Õ¤äàÒOåÐ1ÑÒ?ÓÔã Ñcã ÑÒ¢ã¤Ù5Ô“Ú-Ö)âØÔOâÜdâÞOÙÔOÛ)؈ޓ٠ÖCã Ñàã5ÔOê‚ÒVÕMã Ñ©Ù ÞˆäJ؈Ñ[ã ÑJÒ-Õ¤ÒàÞ‡ÜJäJÛ)Ò-Õ¢Ö)Õ?Ö)Û)Û)äÕ@ã¤Ù5Ôã Ò(ÜuÖ)âuó¦Ö)ØJå §‡åã¤Ù5Ô“âÕ¤ÖCã Ö)ÞˆâÓÞOÖ)â‡ãnÞˆâÆã ÑÒcâJÒ-Ö)؈чéÞOÙ Ö)âJػդäJ٤ߗÔOÚ-ÒOå ä©ÙÕ@Ý©Õ@ã Ò-ààAÔ“Ö6â©èž «-±¨žy£‚«.Â-£‚¤5žVª¤-³r¢,£¨Å-¯)ždßÃÞOÙ·Ò(ÔOÚ5Ñ`Õ¤ä©Ù¤ß,Ô“Ú-Òˆå3Ð1ÑÖ)Õdã5Ô“éÛ)ÒÆÔOÛCèã5Ô“Ö6âJÕAÔOâÒVÙ àÖ)âÔ“ã Ö)ÞOầ ÞOߊã ÑÒXÔOÜ Ô“Ú-Ò-â¨ãAÕ¤äJ٤ߗÔOÚ-ÒÆÔ“Õç1Ò-Û)ÛlÔOÕÛ)Þ(çMÕAÒGÚ-Ö)Ò-â¨ã·ÜJÒeã,H P1¡¤ *—'-§¡@(§J¥¨'()£¦ þnÖ)Õ¤ÚVÙ ÒVã ÒQàÞׂÒ-àÒ-â¨ãÔOÛ)ÞOâØMã ÑÒ§6IÒ-Û6Ô“ã Ò-ÕQã ÞÔÜJÖCÙ Ò-ÚVã Ò-ÜcÛ)Ö6âJÒ¢Õ¤Ò-؈àÒVâ‡ãQÖ)âdÓÔ“Ù5ÔOàÒeã¤Ù Ö6Ú¢Õ¤ÓÔOÚ-ÒOåÕ¤äJ٤ߗÔOÚ-ÒMÚ-Þ“Ù¤ÙÑJÒxÔ“ÓÓ©Ù ÞˆÓJÙ Ö6Ôã Ò%ã¤Ù Ö)ààÖ)âØdÛ)Þ¨ÞOÓÔ“âÜuÒ(ܩ؈Ò%Þˆâ¨ã ÞçMÑÖ)ÚeÑXã ÑJÒÁã¤Ù5Ô“âÕ¤ÖCèãÖ)Þˆâ}Õ¤ÑJÞˆäJÛÜGÞ¨Ú-Ú-äJÙ(åyÐ1ÑJÒÕ¤Ò-؈àÒVâ‡ãxÖ)Õ%ßÃÞˆäJâÜé¨ÝGÖ6âÜJÒVá©Ö)âØXÖ)â¨ã Þ»ã ÑJÒãäJÚVã Ò(Ü|äJÕ¤Ö)âØGã ÑJÒuÚ-ä©Ù¤Ù Ò-â¨ãAÚ-Þˆâ¨ã5ÔOÚeãAÓÞˆÖ)â¨ã ÕÐ1ÑÖ)ÕAÕ¤Ò-؈àÒVâ‡ãdÖ6ÕAÚ-ÞOâÕ@ã¤ÙÖ)Ú»ÚVÞ¨ÞOÙ5ÜJÖ)âÔ“ã Ò-ÕAÔ“âÜ+ã ÑÒ·âÒVá‡ãcÛ6Þ¨Ú-Ô“ã Ö)ÞˆâTÚ-ÔOÛ)Ú-äÛ6Ôã Ò(Ü>äJÕ¤Ö)âØÓÔÙ5ÔOàÒVã¤ÙÞ+ã ÑÒÆÕ¤Ò-ØOàÒ-â¨ãAã ÑÔ“ã[Ú-Þ“Ù¤Ù Ò-Õ¤ÓÞˆâÜJÕ[ã Þ+ã ÑJÒXÞˆâJÒGÖ)â¨ã ÒVÙ Õ¤Ò-ÚVã Ò-ÜåÒ(ܩ؈ÒXãÔOÚVÒ-â¨ãcã¤Ù Ö)ààÖ)âØuÒ(ܩ؈Ò-Õ%Ù äâ}Ö)â}ÞˆÓJÓÞˆÕ¤ÖCã Ò[ÜJÖCÙ Ò-ÚVã Ö)ÞˆâJÕ-ë;ÔOâÜö‡Ö6âJÚ-Ò·ÔˆÜ1Ò-ÚVãlÓÔÙ5ÔOàÒVã¤Ù Ö)Úã¤Ù5ÔOÚVÖ6âJØJå?û´ßã ÑJÖ)ÕŠÕ¤Ò-ØOàÒ-â¨ã(ëÞOÙ%Ÿd O®“žVŸžVª¢1®“ž ¤(¢, ˆ¦5ëÜJÖCÙÒeÙ Õ¤Ò-ÚVã Õ¢Ô“â‡ÝdÞOßã ÑJÒlÕ¤ä©Ù¤ß,Ô“Ú-Ò¡ Õ1ã¤Ù Ö)ààÖ6âJØÁÕ¤Ò-؈àÒ-â¨ã ÕQã ÑÒ-â·ÔÁéÞˆäâÜ©èÖ)â‡ãÑJÒŠâ‡äàxéÒVÙ1ޓߦդÒ-ØOàÒ-â¨ã Õ¢Ö6â·ã ÑÒ-Õ¤ÒŠÒ(ÜJØOÒ-Õ¢Ö)Õ1ã ÑÒnÕ Ô“àÒlß—Þ“Ù?éÞOã ѻդä©Ù¤èãÑJÒÁÖ6âÜJÒVá»ÞOß ã ÑÒÁÓ©Ù ÞˆÓÒVÙlÕ¤Ò-؈àÒVâ‡ãlÚ(Ô“âyéÒßÃÞˆäâÜXÜJÖCÙ Ò-ÚVã ÛCÝuÔOÕß—ÔOÚ-Ò-Õ-ëãÞˆÕ¤Õ¤Ò(Ü&å%Ð1ÑÒxÛ6Þ¨Ú-Ô“ã Ö)ÞˆâGÞOßQã ÑÒ%Ö6â¨ã ÒVÙ Õ¤ÒVÚVã Ö)ÞˆâyÖ6ÕnÜJÒVã ÒVÙ¤èÔ“Ù¤ÝXÑÔOÕnéÒVÒ-âÆÚVÙÖ)âJØ»ã ÑÒ%Ö)â‡ã ÒeÙ Õ¤Ò-ÚVã Ö)ÞˆâGÓÞˆÖ)â¨ãÚ-Û)ÞˆÕ¤Ò-Õ@ãÁã ÞAã ÑÒxÚ-äJÙ¤Ù Ò-â¨ãàÖ)âÒ(ÜXé¨ÝXÕ¤Ò-Û)Ò-ÚVãÚ-Þˆâ¨ã5Ô“ÚVã?ÓÞˆÖ)â¨ã(åÑJÒâ‡äàcéÒeÙMÞOßã¤Ù Ö)ààÖ)âØdÕ¤Ò-ØOàÒ-â¨ã ÕMÓÒVÙŠÕ¤ä©Ù¤ß,Ô“Ú-ÒxÚ-ÔOâXéÒö©Ö)âÚVÒÁãÖ)Þˆâ»ÔOÛ)ÞˆâJØcã ÑÖ)Õ1Õ¤Ò-ØOàÒ-â¨ã(ë)(‚ë©Ö)Õ1ã ÑÒVâ[ØOÖ6׈Ò-â[é¨Ý*(‚ì@î!,+ñ çMÑÒeÙ Ò-+»Ö)ÕÕ¤ÖCãÑJÒxÓÒVÙ ÚVÒ-â¨ãÁÔOÛ)ÞˆâJØ[ã ÑJÒxÖ)â¨ã ÒVÙ Õ¤Ò-ÚVã Ò-ÜyÕ¤Ò-ØOàÒ-â¨ãŠã ÑÔ“ãnã ÑJÒxÖ)â¨ã ÒVÙ Õ¤Ò-ÚVã Ö)ÞOâã؈҈ë;ÖCãdÖ)ÕcâJÞOãcÓÞˆÕ¤Õ¤Ö)éÛ)Ò[ã ÞÆÚ5ÑÒ-Ú5êÒVׂÒVÙ¤Ý+Õ¤ÒV؈àÒ-â¨ã%ß—ÞOÙdÖ6â¨ã ÒVÙ¤èׂÒVÙ¤ÝÛÔÙÖ)Þˆâå ä©Ù?Õ¤ÞˆÛ)äJã Ö)ÞˆâÆã Þdã ÑÖ)ÕlÓJÙ ÞˆéJÛ)Ò-à Ö6Õ?ã ÞAޓ׈ÒVÙ Û6Ô(Ý»Ò(ÔOÚ5ÑXÕ¤äJ٤ߗÔOÚ-ÒÕ¤Ò-ÚVãÞ¨Ú-ÚVäJÙ¤Ù Ò(Ü&åA93

âÚ-Òdã ÑJÒdÛ)Þ¨Ú(ÔOÛ;Ú-Û)ÞˆÕ¤Ò-Õ@ãcÓÞˆÖ)â¨ãÁÖ)Õ%ßÃÞˆäâÜyã ÑJÒAÔOÛ)؈ޓ٠Ö)ã ÑJàËÚeÑJÒ-Úeê‡ÕÞGÕ¤ÒVÒuÖCßlã ÑJÒ[ã¤Ù5ÔOÚ5ê‚Ò(Ü}ÓÞOÖ6â¨ãÕ¤ÑJÞˆäJÛÜ+Ù Ò-Û)Ò(Ô“Õ¤ÒußÃÙ ÞOà ã ÑÒ·ã¤Ù Ö)àuå äJÙã2¤£ÁPA2;Dx¢*—Í»Dx¢ÑJÒxÜ©ÖCÙ Ò-ÚVãlÑÔOÓ©ã Ö)ÚÁÙ ÒVâÜJÒeÙ Ö6âJØ·ÔOÛ)؈ޓ٠Ö)ã ÑJàŠÕ¤Ö)âØã,H 0>£¦&§7 AT0|£'()£!¢ÑÒVÙ|ã ÑJÙ ÞˆäJ؈Ñ{àAÔOâJÖ6ÓJäÛ6Ô“ã Ö)ÞOâ!ëyÔOâJÖ)àAÔ“ã Ö)ÞˆâëGÞ“Ù9Ü©Ý©âÔOàÖ)Úø#ÑÒeã Î%Ƨ)£!§# ¢¡¤£!F!§¢âÒMÞOßã ÑÒlÜ©Ù5Ô-çMéÔOÚ5ê©ÕQã ÞÁÓÞˆÖ)â¨ã8Ó©Ù ÞˆéÒ?àÒVã ÑއܩÕ;Ö)Õ8ã ÑJÒlÜ©Ö)àÒ-âJè 0T&7 '( K ¢¡¤£!F!§}=M£!'($¥‡'¢ä©ÙAÚ-äJÙ¤Ù Ò-â¨ãAÖ)àÓÛ)Ò-àÒVâ‡ã5Ôã Ö)Þˆâ>ÞOß%ÜJÖCÙ Ò-Úeã[ÑÔOÓ©ã Ö6Ú[Ù Ò-âÜ©ÒVÙ Ö)âØ+Þ“ßÒŠã ÑÖ)Õ¢ÓÞOÕ¤Õ¤Ö6éJÖ)Û6ÖCã´Ý[ç8ÒÚ-ÞˆàÓJäJã ÒlÔcàÞ‡ÜJÒ-Ûã ÑÔ“ãMÖ)ÕMÓJÙ Þ1¤Ò-Úeã Ò(ܻވä©ã¤èÖ)âÔãÑÒMÙ5ÔˆÜJÖ)äJÕ¢ÞOßã ÑÒnÜ©Ò-Õ¤ÖCÙ Ò(ÜAÓJÙ ÞˆéÒOåQý9àއܩÒ-ÛÞOßã ÑÖ)Õ8ã´Ý©ÓÒlÖ)Õç1Ô“Ù5ÜAé‡Ýcã 2¦§+P8¡ $¥©>$&9›;§7 §J$(§ Ð!Ù5ÔOÚVÖ6âJØ]ÔOÛ)ÞOâØ Ô§6IÖ)à²Ò(ܩ؈ÒuÖ)ÕdÚ-Û)ÞOÕ¤Ò-ÛCÝ>Ù ÒVÛÔã Ò(Ü>ã ÞÆã¤Ù5ÔOÚVÖ6âJØ+Ô“Û6ÞOâØyã ÑÒ»Õ¤ä©Ù¤ß,Ô“Ú-Òˆå|Ð1ÑÒã¤ÙÖCã ÑàoàcäJÕ@ãÁÕ¤Û)ÖÜ©Ò»Ô“Û)ÞˆâØXã ÑJÒÒ(ÜJØOÒ·Ö)âræ äÚ-Û)Ö6ÜJÒ-ÔOâÒ(ÜJØOÒã¤Ù5ÔOÚ-Ö)âØyÔ“Û6ØOÞOÙÞ?Ô¢ÓÞˆÖ)â¨ã¦Û)Þ¨Ú(Ô“Û6ÛCÝÁÚ-Û)ÞOÕ¤Ò;ã Þ¢ã ÑÒQÓJÙ ÞOéÒ-ÕÓÞOÕ¤Ö)ã Ö)ÞOâ!å ä©Ù ¨ Õ¤ÓÔ“Ú-Ò;ãÒ-âÙ ÒVßÃÒVÙ¤Ù Ò(Üdã Þ%ÔOÕ¢Ô“â·Þ“ú&Õ¤ÒVã1àއܩÒ-Ûì,òlÞJëî-ïˆï‚÷ˆñ5å äJÙ;Õ@ÝJÕ@ã ÒVà äJÕ¤Ò-ÕÞ“ßÃãÑJÒnÞOú&Õ¤ÒVã?àއܩÒ-Û&Ö6â[ã ÑÒnÑÔOÓJã Ö)ÚnÙ Ò-âÜJÒVÙ Ö)âJØcÓJÙ Þ¨ÚVÒ-Õ¤Õ?çMÑJÖ)Û6ÒÁäJÕ¤Ö6âJØxã ÑJÒãÖ6ØOÖ)âÔOÛ!àÞ‡ÜJÒ-ÛßÃÞOÙ?ã ÑÒnׇÖ)Õ¤äÔ“ÛÜ©Ö6Õ¤ÓJÛ6Ô(ݨåÞ“ÙÒyð+Ö6Û)Û)äÕ@ã¤Ù5Ôã Ò-Õ[ã ÑJÒÆÚ-ÞˆâJÕ@ã¤Ù äÚVã Ö)Þˆâ9Ô“âǛ äJÕ¤ÒÆÞOßxÔOâ#Þ“ú&Õ¤ÒVãó¦Ö)؈ä©ÙÒ-Ú-Ö)Õ¤Ò-ÛCÝGã ÑÔ“ã(åXÐ1ÑÒAÔOÛ)ØOÞOÙ ÖCã ÑàoÓJÙ Þ1¤Ò-Úeã Õxã ÑÒAÓ©Ù ÞˆéÒàÞ‡ÜJäJÛ6ÒAÜ©Þ¨Ò-Õ%ÓJÙÞxã ÑJÒŠÚ-ä©Ù¤Ù Ò-â¨ã¢Õ¤Ò-ØOàÒ-â¨ã(åû´ß!ã ÑÖ)Õ1ÓJÙ Þ1¤ÒVÚVã Ö)Þˆâ»Ù Ò-àAÔ“Ö)âÕ¢ÞˆâAã ÑÒlÚ-äJÙ¤èÞˆâ¨ãÒ-â¨ã?Õ¤Ò-؈àÒ-â¨ãMã ÑÒVâ»ã ÑÖ)Õ?ÓJÙ Þ1¤Ò-ÚVã Ö)ÞOâyÖ)Õ?Þˆä©ÙlÙ Ò-Õ¤äJÛCã(å1û´ß´ëÑÞ(ç8Ò-ׂÒVÙ(ëJã ÑÒÙÞ1¤Ò-Úeã Ö6ÞOâÖ)ÕéÒVݨވâÜxÒ-ÖCã ÑJÒVÙ Ò-âÜ©ÓÞˆÖ)â¨ãÞOßã ÑÒ1Õ¤Ò-ØOàÒ-â¨ãã ÑÒ-âÁç1Ò¢ÚVÞˆâJèÓJÙÖ)ØOÖ6âÔOÛ;àÞ‡ÜJÒ-ÛÖ)âGó¦Ö6Ø©å¦ðOÔ[Ö)ÕÁÞOú&Õ¤ÒVãÁé¨Ýuã ÑJÒxÙ5ÔOÜJÖ)äÕÞ“ßàއܩÒ-Û,ådÐ1ÑJÒcÞOÙÑJÒnÓJÙ ÞOéÒnÖ)â[ã ÑÒnÜJÖCÙ Ò-ÚVã Ö)Þˆâ»ÞOߦã ÑJÒnÕ¤äJ٤ߗÔOÚ-ÒâJÞOÙ àAÔOÛÙ Ò-Õ¤äJÛCã Ö6âJØdÖ)â[ã ÑJÒãÒ-Ü[Ù Ò-؈Ö)ÞOâÕ?Ô“Ù Ònã¤Ù Ö)ààÒ(ÜAÔ-ç¢Ô-ݨë©ÓJ٠އܩäÚ-Ö)âØàއܩÒ-ÛÖ)âuó¦Ö)ØJåðOéå;û´Õ¤ÞˆÛ6Ô“ãÑJÒuÞOú&Õ¤ÒVã·àއܩÒ-ÛMÖ6âTó¦Ö6Ø©å¢ð“Úˆå4ü8Þˆâ¨ã5ÔOÚeã[çMÖCã Ñ`ã ÑJÒuÕ¤äJ٤ߗÔOÚ-ÒÆÞOߊã ÑÖ)ÕãÖ)â©äJÒ%ã Þ·ÓJÙ Þ1¤ÒVÚVãÁÞˆâ¨ã Þ·Õ¤Ò-؈àÒ-â¨ã ÕŠÔOÛ)ÞˆâJØ[ã ÑÒ%Û)Þ¨ÞˆÓGÖ6âÆã ÑÔãÜ©Ö)Ù ÒVÚVã Ö)ÞˆâãÖ)ÛÔcÛ)Þ¨Ú(ÔOÛ¦àÖ)âJÖ6àxäà Ö)ÕMßÃÞˆäJâÜåäâ¨ãÒ-Ó©Ù Ò-Õ¤Ò-â¨ã ÕxÚVÞˆâ¨ã5ÔOÚVãÁçMÖCã Ñã ÑJÒdÞOÙ Ö)؈Ö)âÔOÛ1àއܩÒ-ÛQçMÖCã Ñ}ÔÞ“ú&Õ¤ÒVã%àÞ‡ÜJÒ-ÛÙÞOéÒ[ì§ó¦Ö)Ø©å&ðOÚ“ñ5å?ÿlÞOã Ö)Ú-ÒÁã ÑÔ“ãŠÔOâ¨Ý»ÓÔ“Ù¤ãlÞOßQã ÑÒÁÞ“ú&Õ¤ÒVãÜ©Ö6àÒVâÕ¤Ö)ÞˆâJÒ(Ü»ÓJÙÖCã Ñà ùJÙ Õ@ãdÒ-×OÔOÛ)äÔ“ã Ò-ÕxÞˆâÒ·ÞOß?ã ÑÒAÕ¤ä©Ù¤ß,ÔOÚVÒ-ÕÔOâÜ+ã ÑÒ-â}äJÕ¤Ò-ÕdÜ©Ö)èÔOÛ)؈ޓÙÒ-ÚVãÓÔ“Ù5Ô“àÒVã¤Ù Ö)Ú·ã¤Ù5Ô“Ú-Ö)âØGã ÞGÜJÒeã ÒVÙ àÖ)âÒã ÑÒ·âÒVá‡ãcÛ6Þ¨Ú-Ô“ã Ö)Þˆâ|Þˆârã ÑÒÙÑÔãMÖ6Õ¢ã¤Ù Ö)ààÒ(ÜAÔ-ç¢Ô-ÝAÙ Ò-ÓJÙ ÒVÕ¤Ò-â¨ã ÕMÔxÓÞOÙ¤ã Ö)ÞOâuÞOߦã ÑJÒŠÞ“Ù Ö6ØOÖ)âÔOÛàއܩÒ-ÛãÑÔã¢Ú-ÞˆäJÛ6Ü[âÞ“ã¢éÒnÚ-ÞOâ¨ã5ÔOÚVã Ò(ÜAçMÖCã Ñ[ã ÑÒŠÜJÖ)àÒ-âJÕ¤Ö6ÞOâÒ(ÜAÓJÙ ÞOéÒˆåàއܩÒ-ÛãÑÔAÓÞˆÖ)â¨ãnÓ©Ù ÞˆéÒdÔ“Û)ÞˆâØ»ÔOâÆÒ(ܩ؈ÒcÚeÙ Ò(Ô“ã Ò(ÜÆé¨Ýuã¤Ù Ö)ààÖ)âØÐ!Ù5Ô“Ú-Ö)âØAçMÖCãÒ-؈Ö)ÞˆâÆÚ-Þ“Ù¤Ù Ò-Õ¤ÓÞˆâÜJÕMã Þã¤Ù5ÔOÚ-Ö)âØàxäÛCã Ö)ÓÛ)ÒÚ-ÞOâ‡ã5Ô“ÚVãlÓÞˆÖ)â¨ã ÕlÞ“ßÔ-ç¢Ô-ÝuÔcÙÑJÒŠÚ(Ô“Û6ÚVäÛ6Ô“ã Ò(Ü·ÓÔÙ5ÔOàÒVã¤Ù Ö)ÚŠÓÞOÖ)â‡ã1Ö6Õ¢ÞOâ·ã ÑJÒlÕ¤ä©Ù¤ß,Ô“Ú-Òì,Ö,å ÒOåÕ¤äJ٤ߗÔOÚ-ÒˆåQûÌß!ãÑÒ%Ú-Þ“Ù¤Ù Ò-ÚVãÕ¤Ö6ÜJÒ%ÞOßQã ÑÒÁã¤Ù Ö)ààÖ)âØÛ)Þ¨ÞˆÓ¡ ·ã ÑÒÁÙ Ö6ØOѨãnÕ¤Ö6ÜJÒxÞOßQã ÑÒÞˆâXãÑÒ-â}ã ÑÒAã¤Ù5ÔOÚVÒ·Ù Ò-Û)Ò(Ô“Õ¤Ò-ÕdßÃÙ ÞOà ã ÑJÒ·ã¤Ù Ö)àuå+ûÌßMã ÑÒÚ-Û)ÞˆÕ¤Ò-Õ@ãAÕ¤ÒV؈àÒ-â¨ãVñŠãÒ-Û)Ò(Ô“Õ¤ÒAÞˆâ¨ã Þuã ÑÒcùJÙ Õ@ã%Õ¤äJÙ¤ß,Ô“Ú-Òã ÑÒ-âyã ÑÒÕ¤Ò-ÚVÞˆâÜGÖ6Õã¤Ù5ÔOÚ-Ò·Ü©Þ¨Ò-ÕÁâÞOãÁÙã ÑJÒnÞOÙ Ö)؈Ö)âÔOÛàއܩÒ-Û&çMÖ)ã ÑuÔdÜJÖ)àÒ-âJÕ¤Ö6ÞOâÒ(Ü·ÓJÙ ÞOéÒdì§ó¦Ö)ØJåð“Ú“ñ5åÑJÒAã¤Ù5Ô“Ú-Ò[ÜJÞ¨Ò-ÕxâÞOã%Ù Ò-Û)Ò(Ô“Õ¤Ò»Þˆâ¨ã ÞÆÒ-ÖCã ÑÒeÙcÕ¤ä©Ù¤ß,ÔOÚVÒˆëÚeÑJÒ-Ú5ê‚Ò(ÜåXøTÑÒ-â+ãÑÒ-â»ÔxÕ¤ÓÒ-ÚVÖÔ“ÛâJÞOÙ àAÔOÛ&Ö)ÕMÚ-ÞˆàÓJäJã Ò(ÜAÜ©Ö)Ù ÒVÚVã Ò(Ü[ßÙ Þˆà ã ÑÒlÓJÙ ÞOéÒ¡ Õ?Û)Þ“èãÖ)ÞOâuã Þdã ÑÒÛ)Þ¨Ú(ÔOÛÚ-Û)ÞˆÕ¤Ò-Õ@ãŠÓÞˆÖ)â¨ã(å¢Ð1ÑÖ)Õ?âÞ“Ù àAÔOÛÒ-âÕ¤ä©Ù Ò-ÕŠÔdÕ¤àÞ¨Þ“ã ÑÚ(Ô“ãÑÒÁÒ(ܩ؈Ò%ÔOâÜXÔOÛ)դ޻ܩÒVã ÒVÙ ÕMß—ä©Ù¤ã ÑÒVÙlàÞׂÒ-àÒ-â¨ãMÖ)â¨ã Þdã ÑÒã¤Ù5ÔOÚ-ÒxÔOÛ)ÞOâØãàÞ‡ÜJÒ-Û,åÔ“ÕnÔéÛ6Ô“Úeê[éÞáÖCã.ݨëÕ¤ÒVׂÒVÙ5ÔOÛ¦ç8ÞOÙ¤ã ѨçMÑÖ)Û)Ò%ÒVá‡ã Ò-âÕ¤Ö)ÞˆâJÕnÔÙ ÒcÓÞˆÕ¤Õ¤Ö)éJÛ6ÒOåløÒxÑÔ׈Ò%Ö)àdèÒ-â¨ãÒ(ÜàއܩÒ-ÛàAÔOâJÖ)ÓäÛ6Ôã Ö6ÞOâ!ë©Ü©Ö6àÒVâÕ¤Ö)ÞˆâJÒ(ÜÓJÙ ÞˆéÒVÕ-ëÔ“âÜAàcäJÛ)ã ÖCèÓÛ)Ò-àÒ-â¨ãì§Ô‚ñ ì,é&ñ ì,Ú“ñ §¦6( )04#( 8 $1"# ©¨ # ¦ ( $ # 30 8

ÖCú&âÒ-Õ¤Õ¦Þ“ß𨧡§¨§&GåQÐ1ÑJÒ;¥ ÑÔ“â‡ã ÞOà Ö)ÕÔ¢Û)Þ(ç}Ö)âÒVÙ¤ã Ö6Ô?ÜJÒ-ׇÖ)Ú-Ò;çMÖCã Ñ ¢Õ@ãÒ-Ò-Õ;ÞOßßÃÙ Ò-Ò(Ü©Þˆà ÔOâÜÔÙ5Ôã ÑÒVÙ8Õ¤Ö)àÓÛ)Ò?Ü©Ý©âÔ“àÖ6ÚVÕQÕ@ã¤Ù äJÚVã äJÙ ÒOå Ð1ÑÒÜJÒ-ؓ٢ ¥¨§©å §¡§ ˆï©å ï¥ î¢ ¢©å ¢¡íŠÞˆéJÛ6Òe㥨 î(ð¡ ‡å)î ÷¥‡å § ¥ ð îˆå ¨÷¤;Ù5ÔOêˆÒ¥¥ î ¥¨§“ð©å ¥ˆ÷ ï ¥‡å)îOî î§ Jå § §íŠÒ-Ô“Ù÷¨¢ Jî ¥¨å §¡§ ˆï©å §“ð ¢ˆð ¢©å ¢¡§ü;Ù5ÔOâJêÑJÒ8Õ¤Ö)àcäJÛ6Ô“ã Ö)ÞˆâxÓJÙ Þ¨Ú-Ò-Õ¤ÕÙ5Ô“âxÞOâdÔOâ ÚVã5ÔOâJÒ;çMÖCã ÑdÜ©äÔ“ÛE¢Á§¡§¨§Ú(Ô“Õ¤Ò-Õ-ëˆãÞ¨Ú-Ò-Õ¤Õ¤ÞOÙ ÕlÔOâÜ ¢?Ò(ÔOÛ)ÖCã.ÝJæ âØOÖ6âJÒ # ØOÙ5Ô“ÓÑÖ)Ú-Õ-åÓ©Ù) §¥1 243 5622(Õ-å)8 ÏnPÏc›l2 ÇÍ»› 7Ò-Õ¤Ò(ÔÙ ÚeÑTÖ)Õã ÞÓ©Ù Þ‡ÜJäJÚ-ÒXÔׇÖCÙ¤ã äÔ“Û?ÒVâ©×‡ÖCÙ Þˆâ©èÐ1ÑJÒu؈ވÔOÛŠÞ“ßÞˆä©ÙÙü8ÞOÛ6Û)Ö)Õ¤Ö)ÞˆâyÚ-Þˆâ¨ã5Ô“ÚVãÔ“âܻ٠Ò-Õ¤ÓÞOâÕ¤ÒÁß—ÞOÙŠàÞ‡ÜJÒ-ÛÖ)àÓÔ“ÚVã ÕlÖ)ÕŠÔ·Ü©ÖGdè9ÓJÙ ÞˆéJÛ)Ò-àuëˆÒVÕ¤ÓÒ-Ú-Ö6ÔOÛ)ÛCÝcçMÑJÒ-âcäJÕ¤Ö6âJØŠã¤Ù Ö)ààÒ(ÜÁÿ£ £¢¡¤¢öëéäJãQÔÚ-äJÛCãö‡äJÙ¤ß,Ô“Ú-Ò·ÓJÙ ÞˆÓÒeÙ¤ã Ö6ÒVÕcÕ¤äÚ5Ñ>ÔOÕ%ã ÒVá‡ã äJÙ ÒAÚ(ÔOâ>ÔOÜÜ+ã ÞXã ÑÒAÙ Ò(ÔOÛ)Ö)Õ¤à9ÑÒ%ã¤Ù5ÔOÚ-Ö)âJØ»ÒVá©ÓÒVÙ Ö)Ò-âJÚ-Òˆåcö©Þ“ßÃã%ÔOâÜGÜ©ÒVß—Þ“Ù àAÔOéJÛ6ÒxàÞ‡ÜJÒ-Û)ÕÁÔÙ ÒÞ“ß8㣢¡¤¢ö&åxÐ1ÑÖ)Õ|àÞ‡ÜJÒ-ÛCèÌéÔ“Õ¤Ò(Ü ÔOÓJÓJÙ ÞˆÔOÚ5Ñ ÓÒVÙ àÖCã Õ>àÞ“Ù Ò#Ú-ÞˆàÓJÛ)ÒVáÿ¡ÞyéÒAÑÔOÓ©ã Ö)Ú(ÔOÛ)ÛCÝ}Ù Ò-âÜ©ÒVÙ Ò(Ü&å+ýlÜÜJÖCã Ö)Þˆâ©èÕ¤Ú-ÒVâÒ-ÕcÔOÕcç8Ò-Û)ÛMÔOÕdàއܩÒ-Û)Õ%ãÒeÙ¢äÕ¤ÒVÙ8Ú-Þˆâ¨ã¤Ù ÞOÛÞ“ß!ã ÑJÒlàއܩÒ-Û)Õ¢ÔOâÜdã ÑÒØOÙ5Ô“Õ¤ÓÕMÔOÕ;ç1Ò-Û)ÛÔOÕ¢Ô“Û)Û6Þ(ç9éÒVã¤ãÞOâàÒ-â¨ãì©£uÔ“Ò-êOÔ(ç1ÔÔOâÜ[òŠÞOÛ)Û6ÒeÙ éÔOÚ5Ñ!ëî(ïOï¡ ‚ñ5åÒ-â‡×‡Ö)Ù¤ ! "$#&%('*) ¤+-, ¤ +/.P% ›l21ÏäJÙdã¤Ù5Ô“Ú-ÒÆÔOÛ)ØOÞOÙ ÖCã ÑàÖ)ÕAàÞ‡ÜJÒ-ÛCèÌéÔ“Õ¤Ò(Ü}çMÖ)ã Ñ`ÔGâJÒ(Ô“ÙAÚ-ÞOâÕ@ã5Ô“â‡ãÖ)àÒ[Ø“Ù Ö܇èÌéÔOÕ¤Ò-Ürã¤Ù5Ô“âÕ¤ÖCã Ö)ÞˆâÖ)âJØ+ÔOÛ)ØOÞOÙ ÖCã ÑàuårÐ1ÑJÖ)ÕdÙ Ò-Õ¤äÛCã ÕdÖ)â>Ò1F¨äÔOÛãàAÔOâJÚ-ÒÆÔOâÜTÔOÚ-ÚVäJÙ5ÔOÚeÝ|ßÃÞOÙ[Ò(ÔOÚ5ÑTàÞ‡ÜJÒVÛ§ë¢Ù ÒVØ‚Ô“Ù5Ü©Û)Ò-Õ¤Õ»Þ“ßÚ-ÞˆàdèÓÒV٤ߗޓÙÓÛ)ÒVá©ÖCã.Ýyì§ó¦Ö)ØJå÷‚ñ5å¢ ( D(%¦ 36¦ )53 $ 8 $1"# 3

äÒVÙ Ö)Ò-Õ%óÞ“Ù ¥¦ÞOÛCÝJØOÞˆâÔOÛ1Ô“âÜ ¥ÔÙ5ÔOàÒVã¤Ù Ö)Ú £»Þ‡Ü©Ò-Û)Õ-ë ÐÒ-Ú5ÑâJÖ6Ú-ÔOÛ ¢lÒVè4£ ü¢ö¨è´ï‚÷è §¨§¡¢©ë£ ŠâJÖ6׈ÒVÙ Õ¤ÖCã.Ý#Þ“ß ?ã5ÔOÑ!ëlþnÒ-ÓÔ“Ù¤ã àÒ-â¨ã·ÞOßdü8ÞˆàdèÓÞOÙ¤ã£GåQÔOâÜröJÔ“Û6Ö)Õ¤éJäJ٤ݨë%¥å då)ëMî(ïOï¨Jë 5Ð1ÑJÒ ¥ òný?ÿnÐÞA££uÔ“Õ¤Õ¤Ö6ÒOë1ЊåÖ)Úlû.â¨ã ÒVÙ¤ß,Ô“Ú-Ò;ý9þnÒ-ׇÖ)Ú-ÒŠß—Þ“Ù¥ Ù ÞOéÖ)âØ©ŠÖCÙ¤ã äÔ“Û é @Ò-ÚVã Õ-ë Ö)â2ˆ¦.«òŠÔOÓJãÖ)Ò-؈Û,ëQå¦ÔOâÜÐ1Ö6Û)Û)ÒVÙ(ë ø¬å)ë¢î(ïOï §¨ëcœ©žd¹[º¼M½?¾½l(“É(À ¤8ÒVÙ Û)Ö)â!ë¥Ö)âØOÒVÙ(å ö‡ÓJÙÒVÕ¤Ò-â¨ã5Ô“ã Ö)ÞˆâJÕßÃÞOÙòŠÔOÓJã Ö)Ú ŠÖCÙ¤ã äÔOÛ ¥Ù ÞOã Þ“ã.Ý©ÓÖ)âJØJë ŠÖ)âL£¢¦. (¤µH§Š¾§¥¢?Ò-ÓJÙ¥1ª¨±ˆ©Ãªž žV¦e©Ãª¨±œž ¤e¨ª©,¤ £‚¯< ˆª(­-žV¦@žVª¤5ž5ÀöJÔ“ÚVÙ5ÔOàÒ-â¨ã Þ©ëü1ýÁå0%žV¡5©C±ˆª ßO¦¤¥!©—¦e¢—¥©£‚¯6¥¢ªJ®(©—¦@ ˆªJŸžVª¢¦£ˆª«[œž-¯)ž¤ e¨JžV¦.£‚¢, ˆ¦?¾³“¡e¸#£e¨&¢—©,¤1eª¢§žV¦Ã­V£‚¤5žV¡&­V¢§žVŸx¡¤Àý?âÔ“ÑÒ-Ö)àuëJü1ý%åÒ-âÜ©Ò(ÜGÖ)â}Þ“Ù5ÜJÒVÙÁã ÞXÓÒVÙ àÖCãÁàÞ‡ÜJÒVÛ;àAÔ“âÖ)ÓäJÛÔã Ö)Þˆâ!ëã¤Ù5ÔOÚ-Ö)âØXé¨ÝGÔÒVá‡ãÞˆéÒˆëQÔ“âÜ+àxäÛCã ÖCè´Ó©Ù ÞˆéÒÚ-ÞOâ¨ã5ÔOÚVã(åGó¦Ö)âÔ“Û6ÛCݨë ã ÑÒ·Ü©Ö6Õ@èÜJÖ)àÒ-âJÕ¤Ö6ÞOâÒ(Ü+Ó©ÙÖ)éäJã Ò-ÜdÕ@ÝJÕ@ã ÒVà Ü©Ò-Õ¤Ö)؈â·ÔOÛ)Û)Þ(çMÕ¢ÑÖ)؈ÑAäJÓ&ÜÔ“ã Ò¢Ù5Ôã Ò-Õ8ÞOâéÞOã ÑAÕ¤Ö6ÜJÒ-Õ;ÞOßã¤ÙÑÒ Õ@Ý©Õ@ã Ò-àuë(Ù Ò-Õ¤äJÛ)ã Ö)âJØ?Ö)âÁÔ“âÖ6â¨ã ÒVÙ5Ô“ÚVã Ö)ׂÒQשÖ)Õ¤äÔOۨܩÖ6Õ¤ÓJÛ6Ô(ÝŠÚ-ÞˆäJÓÛ)Ò(ÜŠçMÖCã ÑãéÔOÚ5Ñ!ë¥å £Gå6ë8î(ïˆï¨ ©ë ¤ònÔ“ÓJã Ö)ÚdþnÖ)Õ¤ÓÛ6Ô-Ý£uÔ“Ò-êˆÔ-ç1Ô©ë¦ò%å!ÔOâÜyòŠÞOÛ6Û)ÒVÙ é @Ò-ÚVã;ílÙ5Ô“Õ¤ÓÖ)âØÔ“âÜ £uÔOâÖ)ÓJäÛ6Ô“ã Ö)âØŠÖ6â ŠÖCÙ¤ã äÔ“Û©æQâ‡×‡ÖCÙ ÞˆâàÒVâ‡ã(ë ßÃÞOÙÔOâXÔ“Ú-Ú-ä©Ù5Ô“ã ÒÑÔ“ÓJã Ö)ÚnÙ Ò-âÜJÖCã Ö)ÞˆâuÓ©Ù Þ‡ÜJäÚVÒ(Ü·ßÙ Þˆà(¤µ ¥ ¥ ¥`eª¢,¯)µ6 ˆª-­-µ&¼M ‚Å ˆ¢—©,¤-¡78§?¥©¢, ˆŸd£‚¢—©— ˆª©À9Ò-äׂÒVâ!ë ¤8Ò-ÛCèÖ)â+£1¦@¥ §Oðˆðè ¥¨§ˆ÷¢©åØOÖ6äJàuë©ÓÓ!åã ÑÒnÔOÚVã äÔOÛàއܩÒ-Û,å£uÔÙ ÑÒVßÃêˆÔ‡ëþcå ø¬å)ëÔ“âÜ Ù Ö)â!ëdþxå æ¢å)ë»î-ïˆïˆð‡ë eö‡Ö6àxäÛ6Ô“ã Ö)ÞOâ2;Î d092;DxP%/= cDcÍü8ÞOâ‡ã5Ô“ÚVã ŠÕ¤Ö)âØ+ýÿŠÞOâÛ)Ö)âÒ(ÔÙ»þnÔOàÓÖ)âJØ £»Þ‡ÜJÒVÛ§ë |Ö6â8£¢¦. (¤µleª¢§žV¦V¸ÑJÞOÙ Õ?ç1ÞˆäJÛ6ÜyÛ)Ö)ê‚Òxã Þã ÑÔOâê[ã ÑÒ ¤8Ö)ÞOÙ ÞˆéÞ“ã Ö)Ú-ÕØOÙ ÞOäÓÆß—ÞOÙÐ1ÑÒcÔOäJãÒŠÔ“âÜdÕ¤ÒVã äJÓdã ÑÒ¢Ù ÞˆéÞ“ã Ö6ÚŠÜJÒ-ׇÖ)Ú-Ò-Õ-ë¨ã ÑJÒ-ÖCÙ8×OÔ“Ù Ö)ÞOäÕ8ÚVÞˆâJèÑÒ-Û)ÓJÖ6âJØÁÞˆÓÒVÙ5Ô㈪£‚¯ ˆª(­VžV¦¤žVª¤5žX ˆª}¼M ¨Å¤ ‚¢—©—¤-¡»£ˆª«§?ª©ÃŸd£‚¢—©, Oª©À £»Ö)ââJÒ(ÔOÓÞOÛ)Ö6Õ-몣‚¢—©,¥è î-ðˆð¡ ‡å£»Ö)ââJÒ-Õ¤ÞOã5Ô‡ëÓJÓ!å&î(ðOðÞˆÛ1Õ@Ý©Õ@ã Ò-àÕ-ëQÔ“âÜ+ã ÑÒAâJÒVã.ç8ÞOÙ ê‡Ö)âØÆÕ¤ÞOßã.ç¢ÔÙ Òˆå}Ð1ÑÔOâJê©ÕxÔOÛ)Õ¤Þy؈ÞXã Þã¤ÙÑÒuÕ@ã äÜJÒ-â¨ã ÕÔ“âÜrÕ@ã5Ô“ú3ÞOßlã ÑÒÆíŠþü ÓJÙ Þ1¤Ò-ÚVã(ë8çMÖ)ã ÑJÖ)ầ çMÑÖ)Ú5Ñ>ã ÑJÖ6Õãê&ë&ø¬å ¢Áå)ë ¢lÔOâÜJÞˆÛ)ÓJÑ!ëö&å ülå)ë¦ó¦Ö)âJÚeÑë£Gå)ë 8Ô“â QÒVÙ¤ã Ñë.¥å £Gå6ë£uÔÙ£Gå)ë©î(ïOïˆð©ë 5ýlÜÜ©Ö6âJØ?óÞOÙ Ú-Ò;óÒ-Ò-ÜJéÔ“ÚeêŠÐÞlílÙ5ÔOÓJÑÖ)Ú-ÕÔ“âÜЦÔ-Ý©Û)ÞOÙû¤û¤û5ë¢Áåê[ç¢Ô“ÕŠÜ©Ò-ׂÒ-Û)ÞOÓÒ(Üå¢ö©äÓJÓÞOÙ¤ãMß—Þ“Ùlã ÑJÖ)Õ?Ù Ò-Õ¤Ò-Ô“Ù Ú5Ñuç¢ÔOÕlÓJÙ ÞׇÖ6ÜJÒ(Ü»é¨Ýç1Þ“Ù£»û0¥è´ï¨ ¥¨§¡¢¨§¥¨ëé¨Ý[þŠý¢£¥&ý4Ø“Ù5ÔOâ¨ãó%¢¨¢ˆð‡î§è´ïOð“è.ü;è §O𠥨îˆëÿö©ó`ílÙ5Ô“â‡ãÒVàÕ1û.Õ¤Õ¤äJÒ-Õý?âÜXö‡ÞˆÛ)äJã Ö)ÞˆâJÕ-ë »Ö)⣢¦. (¤µ;¾‡: ¼;§¢£-ÀÿlÒVçö¨ÝJÕ@ãÙ Û)Ò(Ô“âÕ-ë©ý%ëJÓÓ塨÷(è© §¥¨åÑÒ[ÿö©ó ÔOâÜ>þlý¡¢¡¥&ý{ö©Ú-Ö)Ò-âJÚ-ÒuÔ“âÜ>ÐÒ-Ú5ÑâÞOÛ)ÞˆØOÝ`ü8Ò-â¨ã ÒVÙÔOâÜ>é¨Ý}ãÒVÙÆílÙ5Ô“ÓÑJÖ6ÚVÕXÔOâÜ ö‡Ú-Ö)Ò-â¨ã Ö)ùÚ lÖ6Õ¤äÔOÛ)Ö*H(Ô“ã Ö)ÞOâ ì§ýnöJü;è ˆï“èß—Þ“ÙGü8ÞˆàÓä©ãO¦%¥!©Ã¦V¢—¥©£‚¯¢¥¢ªJ®(©—¦@ OªŸžVª¢d£ˆª«§?ªJª¥©£‚¯M¾³“Ÿl¨µ¢#£e¨&¢—©,¤ueª¢§žV¦Ã­V£‚¤5žV¡Á­Ve¨JžV¦.£‚¢, ˆ¦%¾³“¡5¢§žVŸc¡@À¦ü8ÑÖ)Ú(ԓ؈ÞJëû@QëJþÁöJü;è QÞˆÛîOëÓJÓ!å ¥Oï¨§è ¢¡§‡îˆåœž-¯)ž¤¥¨§¥‡î(ï‚ñ5å8 2;›Š2;Dd=2 ›Š25ÐÞOäÚ5Ñ4ý?âÜTÐÙ5ÔOÚ-Ò â#Ð1ÑÒGóJÙ Ò-Òeè.óJÞOÙ àýŠÜJÔOÚ5ÑÖ,뢡cå)ëÁî(ïˆï¨¢©ë äÑJèA¡;ÞˆäJâØJë£Gå)ë;ö‡ã Ò-ÒVÛ6ÒOë£Gå)ë ¤;Ù Þ¨Þˆê‡Õ-ëQó8å ¥¦å¥ˆÙ(å)룻Ö)âÕ¤ê¨Ý¨ë%£Gå6ë5óÒ-ÒVÛ6Ö)âJØXýlâÜö‡Ò-Ò-Ö)âبnû.Õ¤Õ¤äJÒ-Õû.âGóÞ“Ù Ú-ÒdþnÖ)Õ@è¤8ÒVÑÒ-âJÕ¤ê‡Ý¨ë3£Gå)ë;î(ïˆï¨§©ë ß/ŠÖCÙ¤ã äÔOÛ é @Ò-ÚVã(ë Ö)⤣1¦@ (¤µ¦¥!©—¦e¢—¥©£‚¯¨¼Mž¤£‚¯ ©,¢—³¨§?ªJª¥©£‚¯ˆeª¢,¯)µö©ä©Ù¤ß,ÔOÚVÒÛ)ÒˆëJøyýÁëÓJÓ!åî(ð¥è î(𨠩徳“Ÿl¨µCÀö©Ò(Ôã¤ãyÖ6â*£1¦@ (¤µM¾³“Ÿl¨© ˆ¡ ©—¥‡Ÿ Oª}eª¢§žV¦@£‚¤(¢—©—®“ž+2B0C1¦.£e¨‡©,¤-¡@ÀŠö‡âÞ(ç¢èÓJÛÔ-ݨënЊëJÓÓ!å ¥¢ §è ¥7A¢©åéJÖ)Ù5Ü&ëE ؈Ö)âJÞJëdå)ëî-ïˆï¨§‡ë û´â¨ã ÒVÙ àÒ(ÜJÖCèýŠÜJÔOÚ5ÑÖ,ë©¡xå6ë©äàAÔ“âÞJëЊå)ë&ÔOâÜÒ ¢?Ò-ÓJÙ ÒVÕ¤Ò-â¨ã5Ô“ã Ö)ÞˆâóÞOÙcö‡ã ÖCú lÖ)Ù¤ã äÔOÛ é ¤Ò-ÚVã Õ-ë ÆÖ)â£1¦@ (¤µ¥©—¦V¢—¥©£‚¯Ô“ãOï©ë þnÒ-Õ¤Ö)ØOâxÐ!Þ¨ÞˆÛ)Õ¦óÞ“Ù ö‡ÑÔOÓJÖ)âØ?ö©ÓJÛ)Ö6âJÒ £»Þ‡Ü©è¢?Ö)Ò-Õ¤Ò-âJßÃÒ-Û6Üë¢Áå6ë¨î-ï¡ŠÖ)â §y£ˆ¢JžVŸd£‚¢—©—¤5£ˆ¯ §yž-¢©(«ˆ¡Q©—ªD OŸl¨¥©¢§žV¦

DSC-3-4HAPTIC RENDERING OF SURFACE-TO-SURFACE SCULPTED MODELINTERACTIONDonald D. Nelson David E. Johnson Elaine CohenUniversity of Utah, Computer Science Dept.50 S Central Campus Dr Rm 3190Salt Lake City, UT 84112-9205Email: {dnelson,dejohnso,cohen}@cs.utah.eduABSTRACTPrevious work in haptics surface tracing for virtual prototypingand surface design applications has used a point model for virtualfinger-surface interaction. We extend this tracing method forsurface-to-surface interactions. A straightforward extension of thepoint-surface formulation to surface-surface can yield extraneous,undesirable solutions, although we rework the formulation to yieldmore satisfactory solutions. Additionally, we derive an alternativenovel velocity formulation for use in a surface-surface tracingparadigm that exhibits additional stability beyond the Newtonmethods. Both methods require evaluating the surface point andfirst and second surface partial derivatives for both surfaces, anefficient kilohertz rate computation. These methods are integratedinto a three step tracking process that uses a global minimum distancemethod, the local Newton formulation, and the new velocityformulation.Figure 2: Virtual proxies are important in the penetrating case.The maximal distance is required by the haptics tracing algorithm.Global solution discontinuities such as “chopping though” an objectare not desirable. Because the velocity formulation (shown) isthe “most local,” it is the method of choice for the penetrating case.1 INTRODUCTIONFigure 1: Well-behaved finger penetration into a surface shown bythe “penetration cylinder”. The velocity method and modified Newtonmethod return the maximum distance between the two surfacesupon penetration.User interactions with surfaces with force feedback are an importantdesign and visualization tool. Current work has mostly utilizeda point position hand model for interaction, but a desirableextension is to allow more realistic geometry for the hand model.However, this extension creates some severe geometric computationchallenges, as well much more complicated contact and responsescenarios.In this paper, we address one portion of the surface-surface hapticrendering problem, namely, computation of proper penetrationdepth between two surfaces. Once the penetration vector has beenobtained, rendering force feedback will be done with establishedA97

haptics techniques [Thompson, 1997].This penetration depth computation will be placed within aframework for reliably finding and tracking multiple contact pointsbetween models. This framework breaks the haptic rendering probleminto several phases — distant tracking using global minimumdistance methods, nearby tracking using local Newton methods,and tracking during contact using a reformulated Newton’s methodor a novel velocity formulation.1.1 Distance ExtremaFollowing [Baraff, 1990], the extremal distance may be definedas the minimum distance between the two models when they aredisjoint, zero during tangential contact, and the locally maximumpenetration depth when they are inter-penetrated. This measure isrelated to the minimum translation distance defined by Cameron[Cameron, 1997].When a user touches a virtual surface with his virtual fingermodel, a curve γ(t) embedded on the finger surface f and a curveζ(t) on the embedded on a model surface g define the path of distanceextrema required by the haptics tracing algorithm.distance extrema = ||f(γ(t)) − g(ζ(t))|| (1)Our goal is to find the piecewise continuous curves γ(t) and ζ(t)for penetration depth computations in a real-time haptics tracingenvironment.When the finger model is penetrated into the CAD model, themaximal distance is required for force computations as shown inFig.1. When the finger is not penetrating the surface the curvesγ and ζ may be discontinuous since the distance between surfacemodels is non-differentiable in general. Both surfaces may be moving.A minimal distance measure between surfaces is useful in thiscase for a global monitoring and restarting mechanism.γ( t)ζ( t)Figure 3: Curves of distance extrema embedded into the finger surfacemodel and the CAD model.2 BACKGROUNDThe robotics community has considerable literature in the area offinding the minimal distance between a point and model or betweenmodel and model [Quinlan, 1994, Gilbert, 1988, Lin, 1994]. However,the minimum distance between models is zero during penetration;we desire the penetration depth for haptic force computations.The minimum distance between parametric surfaces f(u, v)and g(s, t) may be described by the following system of equations(f(u, v) − g(s, t)) · f u =0 (2)(f(u, v) − g(s, t)) · f v =0 (3)(f(u, v) − g(s, t)) · g s =0 (4)(f(u, v) − g(s, t)) · g t =0 (5)which correctly describes that the line between the closest pointsis normal to each surface. This system has been solved with asearch over the four-dimensional parameter space [Snyder, 1995],with global, high-dimensional resultant methods [Lin, 1994], andEuclidean space bounding methods [Johnson, 1998a].Note that in haptic applications, we are often interested in the localsolution to distance. Using a local solution constrains the forcecomputation to a continuous solution, similar to a “God-object”[Zilles, 1995] or virtual proxy [Ruspini, 1997] as used in polygonalmethods (see Fig.2).The assumption that the situation upon penetration is local forhaptics tracing is fortunate because the haptics controller often requiresthe greatest update rates precisely at the impact or penetratingevent. Greater stability is achieved with the high update rates.Local solution methods have been applied to point - surfaceanalogs of Equations 2-5. A first-order solution to the minimumdistance is described in [Thompson, 1997] and extended to a Newtonformulation in [Johnson, 1998b, Stewart, 1997]. For a parametricsurface f(u, v) and point P, we can describe the minimumdistance constraint equations as(f(u, v) − P) · f u =0 (6)(f(u, v) − P) · f v =0. (7)Offset surfaces have recently been used to extend the pointsurfacefinger model with a sphere-surface model [Ruspini, 1997,Thompson, 1999]. A surface that is offset from an original surfaceby a constant amount can be traced with the point-surface model.When the original surface is displayed, the point model is effectivelyperforming sphere-surface tracing. A sphere is still a verysimple finger model; rotating the finger has no effect. We developa method in this work that allows more complicated finger models.3 APPROACHWe have broken the haptic rendering of surface-surface interactionsinto three phases. In the first phase, the far phase, a global monitoringmechanism returns all portions of a surface within some distanceof some part of the other surface. In the second phase, thenear phase, local Newton methods determine the closest points betweenall portions of the surfaces returned from the far phase. Thethird phase, the penetration phase, uses these closest points to initiatea velocity formulation that maintains the proper penetrationdepth vector between surfaces.3.1 Global Minimum DistanceThe global minimum distance mechanism depends on the subdivisionproperties of the surface. For NURBS surfaces, the surfaceis made up of a patchwork of polynomial pieces. Each polynomialpiece is contained with the convex hull of its defining control points.These pieces may be refined, such that each piece splits into severalpieces that maintain the original surface shape, yet have additionalcontrol points. These additional control points converge quadraticallyto the surface as the refinement is increased.We can exploit these properties to prune away portions of surfacesthat are further away than some threshold, refine the remainingportions, and repeat [Johnson, 1998a]. Remaining areas may beused to find approximate minimum distances to initiate faster, local,Newton methods.3.2 Local Minimum DistanceThe local minimum distance phase uses Newton methods to quicklyupdate the minimum distance between portions of the model. WeA98

use a reformulated extremal distance approach described in the followingsection for extra stability relative to the standard minimumdistance formulation from the minimum distance formulation ofEquations 2-5.3.3 Local PenetrationWe have developed two methods for maintaining the proper penetrationdepth vector for surface-surface interactions. The first is areformulation of the Newton minimum distance method. The secondis a velocity formulation. The Newton method has the advantageof being able to find the extremal distance given nearby startinglocations on the surfaces. The velocity formulation is able to maintainthe proper extremal distance and shows stability beyond that ofthe reformulated Newton method.3.4 Extremal Distance RepresentationThe minimum distance equations 2-5 are not adequate to properlydescribe the extremal distance needed for surface-surface haptics.Along with a root at the extremal distance, a set of roots occursalong the curve of intersection between the two surfaces, where(f(u, v)−g(s, t)) goes to zero. The local methods may very easily“slide” into these solutions. Our reworked formulations avoid thezero distance roots.The following methods are applicable to any parametric surfacerepresentation, including NURBS and subdivision surfaces, themost commonly used representations. Let u = [ u v s t ] Tdesignate the closest parametric contact coordinates between anytwo parametric surfaces f(u, v) and g(s, t). f and g denote surfaceevaluations, or mappings from parametric space to Cartesian space.3.5 Newton Extremal Distance FormulationThe extremal distance between parametric surfaces f(u, v) andg(s, t) may be described by the following equation:E(u, v, s, t) =(f(u, v) − g(s, t)) · N (8)where N is the surface normal of f at (u, v). We wish to find theextrema of E, which may be found at simultaneous zeros of itspartials. The partials aref u · N +(f(u, v) − g(s, t)) · N u =0 (9)f v · N +(f(u, v) − g(s, t)) · N v =0 (10)−g s · N =0 (11)−g t · N =0. (12)Noting that the normal N is orthogonal to the tangent planeformed by the partials f u and f v, we may remove the f u · N andf v · N terms. Additionally, the partials of N lie in the tangent planeof f. The equivalent constraint may be formulated by replacingthese partials with the partials of f. These substitutions form a simplifiedset of equations.N · g s =0 (13)N · g t =0 (14)(f − g) · f u =0 (15)(f − g) · f v =0. (16)The first two equations constrain the solution to collinear normalsand the second two maintain collinearity of the closest pointswith the surface normals. This set of equations is analogous to thoseused in [Baraff, 1990] and [Snyder, 1995], however, we have expressedthem in a form more suitable to the demands of haptic ratecomputation.This system of equations may be locally solved through incrementalupdates of u using multi-dimensional Newton’s method,∆u = J −1 (−F) (17)where F is the constraint violation defined by Eqs. 13-16, and J isthe Jacobian of F.This formulation may result in extraneous zeros, since there maybe multiple locations where the surfaces’ tangent planes are paralleland are at a local distance extrema. However, these undesired rootsare typically at polar opposites of the model and are less commonthan for the minimum distance formulation.3.6 Velocity Extremal Distance FormulationA different approach in the penetrating case is to take surfacevelocity into account. The relation of parametric contact differentialswith the relative linear and angular velocity betweenthe two surfaces can be used to provide incremental tracing updates.In the Appendix, the authors have extended the results of[Cremer, 1996, Montana, 1986] to arbitrary surface parameterizations,i.e. the contact velocity relations have been extended to surfaceswhose partials are not everywhere perpendicular, making therelations useful for common models. The parametric contact coordinatesu may be integrated through time using the following relation,derived in the Appendix,[ ]vx,y˙u = A(18)ω x,ywhere [v T ,ω T ] T are the relative linear and angular velocity betweenthe surfaces relative to the frame x g = g u/||g u||, z g =g u × g v/||g u × g v||, and y g = z g × x g located at the contactpoint g(u) (Fig.6), and[ ]Rθ (E f − βF f −y,xA =) −1 −Eg−(R θ F f ) −F g (19)[ [ ] ]E f T= xf y f fu (20)[ ]F f T xf z=uy T (21)f z u[ ]cosθ −sinθR θ =(22)−sinθ −cosθwhere θ represents the angle between parametric axes g s and f u,β is the distance between contacts, x f = f u/||f u||, z f = f u ×f v/||f u × f v||, and y f = z f × x f . E g , F g are defined in a similarmanner.The authors have also developed the relation in terms of worldframe quaternion velocity body coordinates in the Appendix, whichmay be more convenient for use in haptics.3.7 Comparing the MethodsThe advantage to using Newton’s iterative method is that it convergesto a solution given a close initial guess. We are required touse the Newton method during the non-penetrating case so that weobtain an exact starting point for the velocity method.The advantage to the velocity space method is that it is an exactrelation at that instant in time; it is not an iterative numericalmethod. The integration of ˙u provides a highly accurate, strictlyA99

continuous tracing update. It is very well conditioned and does notsuffer from the optimization problems of Newton’s method. However,it does not converge to the true minimal distance given onlyan approximate starting point. It is a good algorithm for generatingvirtual proxy information because it is a strictly local distance update;the curves γ(t) and ζ(t) are continuous, where they may bediscontinuous in small intervals with the Newton method.The authors have found that haptics surface tracing should be asminimally confusing as possible. When given a choice of switchingto a non-local point, which would cause discontinuous forcefeedback, it has been our experience that the more local choice isdesirable. This choice is enforced by the continuous updates providedby the velocity method.4 ALGORITHM & IMPLEMENTATIONIn sum, we have the following algorithm for the non-penetratingand penetrating case:• Far : Global distance refinement, obtain approximate u• Near : Newton iteration, obtain exact u• Very Near and Penetrating : Velocity formulation, using exactuDuring tracking (the outside, non-penetration case), we use aglobal “monitoring” mechanism. Newton iteration and the velocitymethod are run concurrently during this case. kilohertz rate updatesare not critical since the haptics control is returning no force duringnon-penetration.Once the Newton or velocity methods detect a penetration, themonitoring and Newton tracking are turned off. It is assumed thatnon-local jumps are not possible because a haptics device can holda user to within .3 mm of the surface. Global jumps are also notdesirable because of the need for virtual proxies for the finger model(see Fig.2).Due to the use of extremal points in tracking and tracing parametricsurfaces, the existing NURBS trimming implementation[Thompson, 1999] may still be used. This model is an approximationfor effects such as falling off or transitioning between edges,because the “middle” of the finger is considered to be completelyoff as soon as the extremal point is off. A full model for tracingedges is considerably more costly in terms of computation and is asubject of future work. However, the important perception of fallingoff edges that is remarkably well rendered with haptic devices is notsignificantly diminished with this approximate model.5 RESULTSOur approach is efficient because two surface evaluations for forthe contact points and surface partial derivatives at the points arerequired for the velocity and Newton formulations. Timing results[Johnson, 1998b] have shown that the point and partial evaluationsare only slightly slower than evaluating only the point. Runningtimes on an SGI R10000 Onyx 2 for two surface evaluations insideAlpha 1 are about .07 milliseconds. Other operations includingthe inverse of the 4 × 4 matrix and other restacking requiredin the velocity and Newton formulation are not insignificant, butrun in .01 milliseconds (that is, the cost of the methods excludingthe surface evaluations). Thus, a single processor system can easilyperform control and surface-surface analysis at several kilohertzupdate rates.Figure 4 shows the concurrent global monitoring and local Newtonand velocity parametric tracing to enable surface-surface contactanalysis at kilohertz rates for haptics control.Figure 4: Combined global monitoring and local parametric tracingenable surface-surface contact analysis at kilohertz rates for hapticscontrol.We have avoided the introduction of unstable artifacts from apurely iterative numerical method at the time of impact that wouldbe felt by the user. The generally accepted noticeable level of vibrationare on the order of less than a micron at various frequenciesfrom 1 Hz to 1kHz [NRC, 1995]. The artifacts caused by the numericalmethods may be many orders of magnitude greater than thejust noticeable error that a user may perceive.Numerical integration of the results from the velocity formulationmay be done with the simple Euler’s method for short intervalsof time in a haptics environment due to the very small step sizesbetween servo loop cycles. For other applications that use largertimesteps, as occurs in our simulation debugging code, we haveused standard fourth order numerical integration techniques. A verylong tracing sequence, on the order of 10 8 seconds for typical usermotions, can be performed in practice with this integration techniquewithout accumulating noticeable errors. Periodic “restarts”due to user transition to the non-penetrating condition and subsequenttracking by Newton’s method occur quite often. Even higherorder integration methods can be employed if some unusual circumstanceor application requires it without excessive cost due tothe efficiency of our tracking techniques.The reliability of these distance methods depends partially onthe underlying stability of the numerical methods. Singular regionsin the case of point-to-surface computations were derivedin [Johnson, 1998b]; we expect similar conditions for the surfaceto-surfaceextremal distance computations. During nearly singularconcave cases, as in Fig.5, the condition numbers during the firstiteration of Newton’s method is roughly 3 or 4, where it has usuallybeen between 1 and 2 in other configurations. The velocitymethod maintains a condition number of roughly 1 or 1.2 for theA100

Thanks go to the students and staff of the Alpha 1 project, withinwhich this work was developed. Support for this research was providedby NSF Grant MIP-9420352, by DARPA grant F33615-96-C-5621, and by the NSF and DARPA Science and Technology Centerfor Computer Graphics and Scientific Visualization (ASC-89-20219).7 APPENDIX: Surface Contact VelocityFormulationA number of different derivations of the kinematics of contacthave been developed in the last 15 years. These formulations relatethe rate of change of the parametric contact coordinates tothe Cartesian velocity and angular velocity of the bodies in contact.Previous works have been limited to surfaces parameterizedhave orthogonal surface partial derivatives. Some are also limitedthe in-contact case[Montana, 1986, Montana, 1988, Murray, 1990,Cai, 1987]. While any surface may be reparameterized to be orthogonal,it may be expensive and impractical to find such a parameterization.The finger may bend in our interacitve application.Finding the reparameterization with full numerical precision is alsoa problem[Maekawa, 1996]. We develop a new derivation for thenot-in-contact, non-orthogonal surface parameterizations.When both surfaces have partials that are everywhere orthogonal,i.e. f u·f v =0and g s·g t =0, ∀u in the parameteric domain, anoriginal result from [Montana, 1986], extended by [Cremer, 1996]for the not in contact case, derives the surface kinematics as˙u f = I f −1 R θ (II g + ĨI f + βII gĨI f ) −1 ([−wyw x]+ II g[vxv y])˙u g = I g −1 (II g + ĨI f + βĨI f II g) −1 ((1 + βĨI f )[−wyw x]− ĨI f[vxv y])Figure 5: The reformulated Newton local distance method (top)may have problems with concave cases that the velocity (bottom)does not have.cases tried so far, for nearly singular and other cases. Because conditionnumbers less than 100 are considered to be small, the lossof precision upon matrix inverse operations is shown to be smallwith both methods. Upon absolute singular configurations, bothmethods have singular matrices. However, Newton’s method hasadditional convergence requirements [McCalla, 1967] that can produceadditional instability in concave regions, even when the conditionnumber is very small. The velocity method does not exhibitthese instabilities. We are investigating more sophisticated numericalmethods to improve the reliability of the Newton method approach.6 CONCLUSIONThe tracing update formulations presented here provide improvedsurface tracing interactions. A fast, compact method for surfacesurfaceupdates for the nearly penetrating and penetrating case havebeen developed and analyzed. The velocity formulation has beenintroduced for use in haptics penetration.ACKNOWLEDGMENTwhere β is the (signed) distance between contacts, 1 is the 2 ×2 identity matrix, the relative linear and angular surface velocitiesof surface g relative to surface f be denoted by v,ω, the surfacecontact velocities be v f ,ω f and v g ,ω g , I is the first fundamentalform and II is the surface curvature or second fundamental form,with subscripts for surfaces f and g. θ[ represents the angle ] betweenparametric axes g s and f cosθ −sinθu,letR θ =and ĨI =−sinθ −cosθR θ IIR θ . The relation may be written in the following matrix form,[ ]vx,y˙u = A . (23)ω x,y7.1 Non-Orthogonal ParameterizationsWe provide a new derivation for A for regular parametric surfaceswhose partials are not orthogonal so the result in Eqn. 23 can beused for typical models constructed by CAD systems.The parametric contact frames in Fig. 6 for the extremal distancecontext are defined with an orthonormal set of vectors. R f =[x f y f z f ] is the rotation matrix from the local contact frame tothe world frame, where x f = f u/||f u||, z f = f u × f v/||f u × f v||,y f = z f × x f . R g is similarly defined. z f and z g are parallel freevectors (Fig. 6).Comparison of the relative surface velocities 1 can be used to re-1 A subscript with x or y such as a x will denote the first component ofthe vector. Several subscripts, such as a −x,y will represent a two-vectorcontaining the negative of the first component and the second componentof a. A superscript T as in a T will denote a vector or matrix transpose.Partitions of matrices may be selected with (row,column) indexing, with“a:b” for a range or “:” for all rows or columns, as in the Matlab TMnotation. The operator extract skew symmetric retrieves 3 independentcomponents from 9 elements of a skew symmetric matrix.A101

Surface ff uy fz f = −z gg uy gSurface giEg_xy=inv(Eg_xy);RphiFf_xy=Rphi*Ff_xy;dRphiFf_xy=d*RphiFf_xy;Fg_xyiEg_xy=Fg_xy*iEg_xy;RphiEf_xy=Rphi*Ef_xy;H=Fg_xyiEg_xy*(RphiEf_xy+dRphiFf_xy)-RphiFf_xy;iH=inv(H);J=iH*[Fg_xyiEg_xy -eye(2)];A=[J; iEg_xy*(RphiEf_xy*J+dRphiFf_xy*J-[eye(2) zeros(2)])];Proof: From Eqs.24,26, we may write˙u g = E g x,y−1 [R θ E f x,y ˙u f + βF f −y,x ˙u f − v]. (30)Figure 6: Closest point surface contact frames. Velocity relationsallow the contact coordinate velocities to be found.late the parametric contact coordinates u f and u g with the linearand angular surface velocities. Let the surface velocities v f and v gand relative surface velocity v be in the frame [ x g y g z g] .In the extremal distance context (see Fig. 6), we havev g x,y + v x,y = R θ v f x,y + βR θ ω f y,−x , (24)ω g y,−x + ωy,−x = −R θω f y,−x . (25)The terms v f and v g will contain ˙u f and ˙u g in the followingequations through due to the chain rule of differentiation. We derivematrices E f x,y and F f x,y for surface f (and analogously for surfaceg) from the relations of linear and angular velocity to separate ˙u fand ˙u g from other terms,[ [ ] ]vx,y f = R T Tf x,y ḟ x,y = xf y f fu2×2˙u f = E f x,y ˙u f ,ω f y,−x = extract skew symmetric(R f T Ṙ f ) y,−x=[ ]T xf z uy T f z u2×2(26)˙u f = F f y,−x ˙u f . (27)Using Eqns. 24,25,26,27, the general non-orthogonal case is reducedto the 4 × 4 system,[ ][ ] [ ]Rθ (E f x,y − βF f −y,x ) −Eg x,y ˙uf vx,y(−R θ F f = .y,−x )x,y −Fg ˙ux,y g ω x,y(28)This system can be solved quickly for ˙u using the followingpseudocode fragment for arbitrary surface parameterizations. Werewrite the inverse of the 4 × 4 coefficient matrix in Eq. 28,[ ]Rθ (E f x,y − βF f −y,xA =) −1 −Eg x,y−(R θ F f (29)y,−x )x,y−Fg x,yto be solved even more efficiently as a series of 2 × 2 matrixinverses and multiplications,Substituting into Eq.25, we haveF g E g x,y−1 [R θ E f x,y ˙u f +βF f −y,x ˙u f −v]+ω = −R θ F f ˙u f . (31)Gathering ˙u f ,F g E g x,y−1 [R θ E f x,y + βF f −y,x + R θF f ] ˙u f = F g E g x,y−1 v − ω.(32)Expressing this equation as a linear system, we haveH ˙u f = F g E g x,y−1 v − ω, (33)for which we can solve for the contact coordinates for surface f,[ ]˙u f = H −1 [F g E g x,y−1 v− 1 2x2] , (34)ωFor convenience, let us representJ 2x4 = H [ ] −1 F g E g x,y −1 − 1 2x2 so that[ ]v˙u f = J , (35)ωNow substituting this solution back into Eq.30,[ ] [ ]˙u g = E g x,y−1 [R θ E f vx,yJ + βF f vω−y,x J − v]. (36)ω= E g x,y −1 [R θ E f x,yJ + βF f −y,x J − [12x2 02x2]] [vωFinally, lettingJ b = E g x,y −1 [R θ E f x,yJ+βF f −y,x J−[12x2](37)02x2]], we may write[ ][ ]J v˙u =. (38)J b ω[ ]JThe matrix A from Eq.29 is , completing the proof. TheJ bsolution is roughly as efficient as previous methods since the inverseof two 2 × 2 matrices and nine matrix multiplications, ratherthan four 2 × 2 matrix inversions and six matrix multiplications, isrequired (see pseudocode fragment). The optimized matrix inverseand multiplication implementation is a constant cost and is a smallfraction of the cost associated with a surface evaluation.A102

7.2 Cartesian and Quaternion Generalized VelocitiesNow we express the relative surface velocities in terms of worldframe body coordinate velocities so that integration of orientationis possible (integration of angular velocity is meaningless). Definethe local contact frame through the rotation matrixR loc = [ x g y g z g] T. (39)Let G f = G(q f,rot ), G g = G(q g,rot), whereG(q rot) is thematrix operator mapping quaternion velocities to angular velocities[Haug, 1992, Shabana, 1998], given by[−qrot 2 q rot 1 q rot 4 −q rot 3]G(q rot) =2 −q rot 3 −q rot 4 q rot 1 q rot 2 . (40)−q rot 4 q rot 3 −q rot 2 q rot 1The velocity [v T ω T ] T is the motion of surface g relative to surfacef. We write our world space velocities in terms of the localframe. We relate relative angular velocity and world frame quaternionvelocity by[ ] [ ]˙qω = R loc −Tf G f T f,rotgG g , (41)˙q g,rotand relative linear velocity to Cartesian velocity throughv = R loc ( ˙q g,tr − ˙q f,tr +(ω glg− ω glf) × (g(u) − qg,tr)). (42)× denotes the vector cross product. T f and T g are rotationsfrom the local frame to the world frame defined by quaternionsq f,rot and q g,rot. We write the relative surface velocities[v T ,ω T ] T as[ ] [ ]v Rloc −R=loc (g(u) − q g,tr)×∗ω 0 R loc[ ][ ]−I3x3 0 3x4 I 3x3 0 3x4 ˙qf(43)0 3x3 −2T f G f 0 3x3 2T gG g ˙q gwhere × in Eq. 43 denotes the 3x3 skew symmetric matrix thatperforms the operation of a cross product [ (obtained from]the three0 −az aycomponents of a vector, i.e. a× =). Fromaz 0 −ax−ay ax 0Eq. 38, the truncated part [vx,y,ω T x,y] T T is all that is required. LetB contain the first two rows and rows four and five of Eq. 43. Substituting[vx,y,ω T x,y] T T into Eq. 38, yields[ ]˙qf˙u = AB . (44)˙q g7.3 Non-Orthogonal Surface-Curve Velocity FormulationSimilarly, it may be shown that the surface-curve extremal distanceequations may be extended to for arbitrary surface parameterizations.The time derivative of the parametric contact coordinates for surfacef and curve g may be written as a function of linear and angularvelocity multiplying a matrix operator,[Rθ (E f u + dF f˙u 3x1 =x,y) −g u x,y−R θ F f x,y −k gsin(φ)g u x,y] −1 [vx,y3x3ω y](45)3x1,wherek g =||gu × guu||||g u|| 3 (46)and φ is the angle between the curve normal (which is −z f forthe extremal distance case) and the curve binormal b, about thecurve x axis g u. The binormal isb =gu × guu||g u × g . (47)uu||Eq. 43 is again used to establish this relation in terms of quaternionand Cartesian velocities, using rows 1,2, and 5.References[Bajaj, 1988] Bajaj, C., Hoffmann, C., Lynch, R., Hopcroft, J.,“Tracing surface intersections,” in Computer Aided GeometricDesign, Vol 5, 1988, pp. 285-307.[Baraff, 1990] Baraff, D., “Curved surfaces and coherence for nonpenetratingrigid body simulation”, in Computer Graphics Proceedings,SIGGRAPH, 24(4): 19-28, 1990.[Cai, 1987] Cai, C., Roth, B., “On the spatial motion of rigid bodieswith point contact,” in International Conference on Roboticsand Automation, pp. 686-695, March 1987.[Cameron, 1997] Cameron, S., “Enhancing GFK: ComputingMinimum and Penetration Distances between Convex Polyhedra,”in International Conference on Robotics and Automation,April 1997.[Cremer, 1996] Anitescu, M., Cremer, J., and Potra, F., “Formulating3D Contact Dynamics Problems,” Mech. Struct. & Mach.,vol. 24, no. 4, pp. 405-437, Nov. 1996.[Cohen, 1980] Cohen, E., Lyche, T., and Riesenfeld, R., “DiscreteB-Splines And Subdivision Techniques In Computer Aided GeometricDesign And Computer Graphics,” Computer Graphicsand Image Processing, Vol 14, Number 2, October 1980.[De, 1998] De, S., Srinivasan, M., “Rapid Rendering of”Tool-Tissue” Interactions in Surgical Simulations: ThinWalled Membrane Models”, PHANToM User’s Group Proceedings,PUG 1998, http://www.sensable.com/ community/PUG98 papers.htm.[Gregory, 1998] Gregory, A., Lin, M., Gottschalk, S., Taylor, R.,“H-Collide: A Framework for Fast and Accurate Collision Detectionfor Haptic Interaction,” UNC Report, currently unpublished,1998.[Gilbert, 1988] Gilbert, E., Johnson, D., Keerthi, S. “A Fast Procedurefor Computing the Distance between Complex Objects inThree-Dimensional Space,” IEEE Journal of Robotics and Automation,pp. 193-203, April 1988.[Haug, 1992] Haug, E., Intermediate Dynamics, Prentice Hall,1992.[Johnson, 1998a] Johnson, D. E., and Cohen, E., “ A frameworkfor efficient minimum distance computations,” Proc. IEEE Intl.Conf. Robotics & Automation, Leuven, Belgium, May 16-21,1998, pp. 3678-3684.A103

[Johnson, 1998b] Johnson, D.E., and Cohen, E., “An improvedmethod for haptic tracing of sculptured surfaces,” Symp. onHaptic Interfaces, ASME International Mechanical EngineeringCongress and Exposition, Anaheim, CA, Nov. 15-20, 1998, inpress.[Kriezis, 1992] Kriezis, G., Patrikalakis, N., Wolder, F., “Topologicaland differential-equation methods for surface intersections,”in Computer-Aided Design, vol. 24, no. 1, January 1992, pp. 41-51.[Lin, 1994] M.C. Lin, D. Manocha, and J. Canny. Fast contact determinationin dynamic environments. in IEEE Conference onRobotics and Automation, pp. 602-609, 1994.[Maekawa, 1996] Maekawa, T., Wolter, F., Patrikalakis, N., “Umbilicsand lines of curvature for shape interrogation,” in ComputerAided Geometric Design, vol. 13, no. 2, March 1996.[Thompson, 1997] Thompson II, T.V., Nelson, D.D., Cohen, E.,and Hollerbach, J.M., “Maneuverable NURBS models within ahaptic virtual environment,” 6th Annual Symp. Haptic Interfacesfor Virtual Environment and Teleoperator Systems, DSC-Vol. 61,(Dallas, TX), pp. 37-44, Nov. 15-21, 1997.[Thompson, 1999] Thompson II, T.V., Cohen, E., “DIRECT HAP-TIC RENDERING OF COMPLEX TRIMMED NURBS MOD-ELS,” in 8th Annual Symp. Haptic Interfaces for Virtual Environmentand Teleoperator Systems, (Nashville, TN), 1999.[Zilles, 1995] Zilles, C.B., and Salisbury, J.K., “A ConstraintbasedGod-object Method For Haptic Display,” in Proc. IEE/RSJInternational Conference on Intelligent Robots and Systems, HumanRobot Interaction, and Cooperative Robots, Vol 3, pp. 146-151, 1995.[McCalla, 1967] McCalla, T., Introduction to Numerical Methodsand FORTRAN Programming, John Wiley & Sons, Inc., 1967.[Montana, 1986] Montana, D., J., 1986, Tactile sensing and thekinematics of contact, Ph.D. Thesis, Division of Applied Sciences,Harvard University.[Montana, 1988] Montana, D., “The Kinematics of Contact andGrasp”, in Int. J. of Rob. Res., vol. 7, no. 3, June 1988.[Murray, 1990] Murray, R., Robotic Control and NonholonomicMotion Planning, PhD. Dissertation, Electronics Research Laboratory,College of Engineering, University of California, Berkeley,1990.[NRC, 1995] National Research Council, Virtual Reality: Scientificand Technological Challenges, Committee on Virtual RealityResearch and Development, National Academy Pres, 1995,p.69.[Quinlan, 1994] Quinlan, S. “Efficient Distance Computation betweenNon-Convex Objects,” IEEE Int. Conference on Roboticsand Automation, pp. 3324-3329, 1994.[Ruspini, 1997] Ruspini, D., Kolarov, K., Khatib, O., “The HapticDisplay of Complex Graphical Environments,” in ComputerGraphics Proceedings, SIGGRAPH, August, 1997.[Ruspini, 1998] Ruspini, D., Khatib, O., “Dynamic Models forHaptic Rendering Systems.” in Advances in Robot Kinematics:ARK’98, June 1998, Strobl/Salzburg, Austria, pp 523-532.[Sato, 1996] Sato, Y., Hirata M., Maruyama T., Arita Y., “EfficientCollision Detection using Fast Distance Calculation: Algorithmsfor Convex and Non-Convex Objects,” in Proceedingsof the 1996 IEEE International Conference on Robotics and Automation,April, 1996, pp. 771-778.[Shabana, 1998] Shabana, A., Dynamics of Multibody Systems,Cambridge University Press, 1998.[Stewart, 1997] Stewart P., Chen Y., Buttolo P., “CAD Data Representationsfor Haptics Virtual Prototyping,” in Proceedings ofDETC ’97, 1997 ASME Design Engineering Technical Conferences,Sept. 14-17, 1997, Sacramento, CA.[Snyder, 1995] Snyder, J., “An Interactive Tool for Placing CurvedSurfaces without Interpenetration,” in Computer Graphics Proceedings,SIGGRAPH, Los Angeles, August 6-11, 1995, pp.209-216.A104

A105

A106

A107

A108

A109

A110

A111

A112

A113

A114

A115

A116

A117

A118

A119

A120

A121

A122

A123

A124

Haptic Rendering—Beyond Visual ComputingToward RealisticHaptic Renderingof Surface TexturesThe emerging science of haptic renderingconsists of delivering properties of physicalobjects through the sense of touch. Owing to therecent development of sophisticated haptic-renderingalgorithms, users can now experience virtual objectsthrough touch in many exciting applications, includingsurgical simulations, virtual prototyping, and data perceptualization.Haptics holds great promise to enrichthe sensory attributes of virtual objects that these systemscan produce.One area that has received increasingattention in the haptics communityis haptic texture rendering, theNew sophisticated hapticrenderingalgorithms let geometry-scale features on objectgoal of which is to introduce micro-surfaces. Haptic objects renderedusers experience virtual without textures usually feel smooth,and sometimes slippery. Appropriateobjects through touch. The haptic textures superimposed on hapticobjects enhance an object’s realism.For example, we can make theauthors systematicallysame cubic structure feel like a brickinvestigate the unrealistic with rough surface textures or a cardboardcarton with finer textures.behavior of virtual haptic Clearly, haptic texture rendering is anexciting research field that can taketextures.haptic rendering to the next level.Although much effort has beendevoted to haptic texture rendering—mostly in modelingand rendering techniques 1,2 —the research communitymust overcome many challenges before haptictexture rendering can be widely used in real applications.One common problem in haptically rendered textures isthat they are sometimes perceived to behave unrealistically,for example, by buzzing or by the apparent alivenessof a textured surface. Due to the complex nature ofthe haptic-rendering pipeline and the human somatosensorysystem, it remains a difficult problem to expose allfactors contributing to such perceptual artifacts.At the Haptic Interface Research Laboratory at PurdueUniversity, we are among the first to have systematicallyinvestigated the unrealistic behavior of virtualSeungmoon Choi and Hong Z. TanPurdue Universityhaptic textures. This article presents a summary of ourrecent work in this area. We hope this article will stimulatefurther discussion among haptics researchers andapplications developers who are interested in haptic texturerendering. Interested readers may refer to our previouspublications for more details. 3-7Perceived instabilityWe use the term perceived instability to refer to all unrealisticsensations—such as buzzing—that cannot beattributed to the physical properties of a textured surfacebeing rendered with a force-feedback device. Todevelop haptic texture-rendering models and methodsthat can deliver realistic textures to human users, youmust understand the conditions under which texturedvirtual objects are free of perceptual artifacts, and youmust also recognize the sources of perceived instabilities.We developed the notion of perceived instability toinclude the effects of all factors in haptic interaction thatcan result in unrealistic sensations. As shown in Figure 1,haptic interaction occurs at a haptic interaction tool thatmechanically connects two symmetric dynamic systems.In principle, each block in the diagram can contribute tothe perception of instability by the human user.A crucial difference between real and virtual hapticinteractions with an object is that a virtual environmentimparts no haptic sensation to the user unless the interactiontool penetrates the object surface. The first phaseof haptic texture rendering is the computation of the penetrationdepth and the resulting force command using ahaptic texture renderer stored in the computer. For thispurpose, most haptic systems repeat several proceduresat a high update rate, usually 1 KHz or higher. First, thesystem measures the position of the haptic interactiontool using position sensors embedded in the haptic interface.The system then compares the measured positionof the interaction tool with the location of objects in thevirtual environment. If the interaction tool penetrates thesurface of any virtual object, a response force is computedand sent to the haptic interface to create the intendedhaptic effect. Finally, if the state of any virtual object haschanged due to the interaction, the system updates theA12540 March/April 2004 Published by the IEEE Computer Society 0272-1716/04/$20.00 © 2004 IEEE

database of virtual objects.Two of these four steps—collisiondetection and response force computation—canhave a significanteffect on perceived instability. Thesetwo steps determine the so-calledenvironment dynamics, the reactiondynamics of the haptic renderer to auser input. In most cases, the environmentdynamics is an approximationof the corresponding real-worldcontact dynamics because simulatingthe actual physics is usually tooHaptic rendering systemComputercomplex to accomplish in real time. The simplified environmentdynamics must preserve the essence of the realcontact dynamics to produce sensations consistent witha user’s experience and expectation. Otherwise, the userperceives unrealistic behaviors of haptically renderedobjects, and perceived instability occurs. This issue hasreceived little attention from the research communitybecause a majority of studies on haptic texture renderinghave focused on the development of time-efficientrendering algorithms.The next phase of haptic texture rendering is the deliveryof force to the human user. During this process, theforce-feedback device must remain stable to avoid perceivedinstability. Device instability, such as mechanicalresonance, can result in force variations in addition tothe force command received from the haptic texture renderer.In our experiences, a user usually judges the hapticallyrendered textured surface as unstable when anextraneous signal—such as high-frequency buzzing—occurs from the haptic interface. This issue has receivedmuch attention in the context of control engineering,although most studies assume a much simpler virtualenvironment such as a flat wall without any textures. 8Much work is needed to extend the techniques for solvingthe hard-wall stability problem.The last phase of haptic texture rendering is the perceptionof force by a human user. The human user sensesthe mechanical stimuli from the haptic interface,extracts information from force variations, forms a perceptof the virtual object being rendered, and determineswhether the virtual object is realistic. To determinewhether the user perceives instability from a textured surfacerendered by the haptic interface, we must resort topsychophysical studies. Psychophysics is a branch of psychologywith well-developed methodology for studyingthe relation between geometrical and physical propertiesof objects and the percept. At this point, little knowledgeis available in the literature on the perceived instabilityof haptically rendered objects, because this is a newresearch topic that has only become relevant with therecent capability to render haptic virtual environments.Goals and approachesOur long-term goal is to develop haptic texture-renderingsystems that deliver realistic sensations of virtualhaptic textures. To do so requires the appropriatedesign of a texture renderer, the stable control of a hapticinterface, and a better understanding of thesomatosensory system. As a first step, our research hasSensingA ctuationHapticinterfaceInteractiontoolMechanically coupledSensorimotorsystemfocused on understanding the nature of perceived instability.Specifically, we■ investigated the conditions under which perceivedinstability of virtual textures occurs,■ discovered the types of perceived instability frequentlyreported by human users,■ identified the proximal stimuli that contributed to theperception of instability, and■ unveiled the sources that produced the stimuli.We conducted psychophysical experiments to quantifythe conditions under which users perceived instabilityfrom virtual textures and to understand the associatedpercept. We also measured the physical stimuli deliveredby the haptic interaction tool to the user’s hand under variousconditions where the textures were perceived to bestable and unstable. By analyzing the measured data, welocated signal components responsible for the perceptionof instability. We achieved the last goal by investigatingwhich component in the haptic texture-rendering systemgenerated the signals that led to perceived instability.Because of the plethora of texture-rendering modelsand methods, we chose a benchmark consisting of themost essential features common to many texture-renderingsystems for studying perceived instability. Inaddition, we had to consider the effect of user explorationpatterns because the user is mechanically coupledto the haptic interface. For the apparatus, we useda Phantom force-reflecting device (from SensAble Technologies)shown in Figure 2. This is the most widely useddevice for haptics research and applications.PerceptionActionHuman userBrain1 Illustration of the key components of the interaction between a haptic rendering system anda human user.Force/torquesensorp(t)yWorld coordinate framexzAccelerometer2 Phantom1.0A used in ourstudies. This is amodified Phantominstrumentedwithadditionalforce-torqueand accelerationsensors.A126IEEE Computer Graphics and Applications 41

Haptic Rendering—Beyond Visual Computingyzxz = Asin(–––2πx) + ALFor the texture model, we used a 1D sinusoidal gratingsuperimposed on a flat surface (see Figure 3). Thegrating is represented byin the Phantom world coordinate frame where A and Lare the amplitude and wavelength of the grating. Sinusoidalgratings have been widely used as basic buildingblocks for textured surfaces in studies on haptic textureperception and as a basis function set for modeling realhaptic textures.For collision detection, we used two methods of computingpenetration depth d(t):andStylus tip: p(t) = (p x (t), p y (t), p z (t))z = A sin ⎛⎝⎜2πL x ⎞⎠⎟ + APenetration depth: d(t)n Wn T (p(t))z = 03 Illustration of the texture rendering models and methods used in ourstudies.⎧0 if pz()t > 0⎪d1()t = ⎨ ⎛ 2πAL p x() t ⎞⎪ sinA p z() t p z()t 0⎝⎜⎠⎟ + − if ≤⎩change in computed penetration depth and responseforce. The disadvantage was that it’s much more difficultto apply this algorithm to textured polygonal objectsfor two reasons. One is the nonlinearity associated withthe representation of textured object surfaces that usuallyrequires iterative numerical algorithms for collisiondetection. The other is the lack of a global representationof the boundaries of the textured virtual objects. In atypical implementation, the polygons and the texturemodel are stored separately, and the texture model islocally mapped onto a point on the polygon whenevernecessary. It’s often infeasible to do a global collisiondetection using only the local information. To the bestof our knowledge, the application of the collision detectionmethod based on d 2(t) to a general class of texturedobjects is still an open research issue.We employed two basic texture-rendering methods.Both used a spring model to calculate the magnitude ofrendered force, but they differed in the way they renderedforce directions. Force magnitudes were calculatedas K ⋅ d(t), where K was the stiffness of the texturedsurface and d(t) was the penetration depth of the stylusat time t (see Figure 3). In terms of force directions, thefirst method rendered a force F mag(t) with a constantdirection normal to the flat wall underlying the texturedsurface. The second method rendered a force F vec(t) withvarying direction such that it remained normal to thelocal microgeometry of the sinusoidal texture model.Mathematically, F mag(t) = Kd(t)n w, and F vec(t) =Kd(t)n T(p(t)), where n w was the normal vector of theunderlying flat wall, and n T(p(t)) was the normal vectorof the textured surface at p(t). Both methods keptthe force vectors in horizontal planes, thereby minimizingthe effect of gravity on rendered forces.The two texture-rendering methods are naturalextensions of virtual wall rendering techniques. Perceptually,they are very different: Textures rendered byF vec(t) feel rougher than those rendered by F mag(t) forthe same texture model. Textures rendered by F vec(t)also feel sticky sometimes.Our exploration mode refers to a stereotypical patternof the motions that a user employs to perceive a certainattribute of objects through haptic interaction. Wetested two exploration modes—free exploration andstroking. In the free exploration mode, users coulddetermine and use the interaction pattern that was mosteffective at discovering the instability of the renderedtextures. We selected this mode as the most challenginginteraction pattern for a haptic texture renderingsystem in terms of perceived stability. With the strokingmode, users should move the stylus laterally across thetextured surfaces. We chose this mode to be representativeof the typical and preferred exploration patternfor accurate texture perception. 9We conducted psychophysical experiments to quantifythe parameter space within which textures were perceivedas stable and to categorize the types of perceivedinstability discovered by users. We employed the methodof limits, a well-established classical psychophysicalmethod, in all our experiments. 10 And we employed adiverse range of experimental conditions, with factorsincluding texture model parameter (amplitude and wave-⎧0if pz() t > h( px()t )⎪d2()t = ⎨ ⎛ 2πAL p x() t ⎞⎪ sinA p z() t p z()⎝⎜⎠⎟ + − if t ≤h(px () t⎩)where p(t) = (p x(t), p y(t), p z(t)) was the position of thePhantom stylus tip and h(p x(t)) = Asin (2π/L p x(t)) + Awas the height of the textured surface at p x(t). The firstmethod, d 1(t), assumed that collision detection wasbased on the plane underlying the textured surface (z =0). The advantage of d 1(t) was that we can easily generalizeit to textured objects with a large number ofunderlying polygons because the plane could representa face on a polygon. The disadvantage was that it introduceddiscontinuity in computed penetration depth(and subsequently in response force) when the Phantomstylus entered and left the textured surfaces.The second method d 2(t) declared a collision as soonas the stylus entered the texture boundary. The advantageof this method was that it ensured a continuous42 March/April 2004A127

0.60.5K T (N/mm)0.40.30.24 Example ofpsychophysicalresults.0.1400.531A (mm)1.5212L (mm)length of the 1D sinusoidal gratings), texture renderingmethod, exploration mode, and collision-detectionmethod. In each experimental condition, a subject’s taskwas to explore a virtual textured plane rendered withPhantom and to decide whether the textured plane exhibitedany perceived instability. The dependent variablemeasured in the experiments was the maximum stiffnessK T under which the rendered textured plane did not containany perceived instability. A more detailed descriptionof experiment design can be found elsewhere. 3,4,6We measured physical stimuli to isolate the signalsresponsible for the perception of instability and to identifytheir sources. For this purpose, we added two moresensors—6D force-torque sensor and 3D accelerometer—tothe Phantom, as shown in Figure 2, and measuredthe nine associated physical variables—3Dposition, 3D force, and 3D acceleration—that were deliveredto a user’s hand. We collected data for many experimentalconditions on the basis of the parameter spaceobtained from the psychophysical experiments. By comparingthe measured data of both perceptually stable andunstable cases in the time and frequency domains, weisolated the physical stimuli that induced the perceptionof instability. We also investigated the sources for thesesignals using additional hypothesis-driven experiments.Parameter spacesFigure 4 shows an example of a parameter space forperceptually stable haptic texture rendering based ondata obtained from the psychophysical experiments. Wemeasured the data when the subject stroked virtual texturesrendered with d 1(t) (collision detection based onthe plane underlying the textured surface) and F vec(t)(variable force directions). In the figure, A and L representthe amplitude and wavelength of the sinusoidaltexture model, respectively, and K T denotes the maximumstiffness value under which a virtual texture feltstable. The blue rectangles in the figure represent thestiffness thresholds averaged over three subjects for thecorresponding texture model parameters. Also shownis a best-fit surface to the measured data found byregression analysis. The region under the mesh surfacerepresents the parameter space of (A, L, K) for perceptuallystable haptic texture rendering, and the regionabove the mesh surface contains parameters that resultin virtual textures that were perceived as unstable.The most significant result of the psychophysicalexperiments was that the parameter spaces for perceptuallystable texture rendering were limited. See Table1 for a summary. Under most experimental conditions,the virtual textures that could be rendered without anyTable 1. Average stiffness thresholds for perceptually stable texture rendering.d 1(t)d 2(t)Experiments Range (N/mm) Mean (N/mm) Range (N/mm) Mean (N/mm)F mag(t), free exploration 0.0586 – 0.1023 0.0799 0.1813 – 0.5383 0.3486F mag(t), stroking 0.4488 – 0.1664 0.3116 0.2490 – 0.6410 0.3603F vec(t), free exploration 0.0097 – 0.0367 0.0209 0.0181 – 0.0260 0.0235F vec(t), stroking 0.0718 – 0.3292 0.1848 0.3254 – 0.4638 0.3808A128IEEE Computer Graphics and Applications 43

Haptic Rendering—Beyond Visual Computingperceived instability felt soft—similar to the feel of corduroy.Textures rendered with higher stiffness valuesusually contained unrealistic sensations, such asbuzzing and aliveness. For the haptic texture renderingsystem used in our experiments to be useful for generatinga large range of textures, we need to enlarge theparameter spaces for perceptually stable rendering.We examined the effects of experiment factors—texturemodel parameters, collision detection method, texturerendering method, and exploration mode—byapplying statistical analysis on the psychophysicalresults. In general, stiffness thresholds tended toincrease when the amplitude of the sinusoidal texturemodel decreased or when the wavelength increased.Collision detection method d 2(t) resulted in larger stiffnessthresholds than d 1(t), except for experiments usingF vec(t) and free exploration where the thresholds weretoo small to exhibit any trends (see Table 1). In mostcases, textures rendered with F mag(t) (constant forcedirection) showed larger stiffness thresholds than thosewith F vec(t) (variable force direction). On average, texturesexplored by stroking resulted in larger stiffnessthresholds than those by free exploration.To gain insight into the effects of texture model parameterson perceived instability, we consider the derivativeof force magnitude. Let g(t) = |F mag(t)| = |F vec(t)|denote force magnitude, and assume that the stylus isin contact with the textured surface. From there we have⎡ ⎛ 2πgt K AL p x t ⎞() = ⎢ sin () A p z()⎝⎜⎠⎟ + − t ⎤⎥⎣⎦Differentiating g(t) with respect to the time variablet results inKA&⎛ 2πgt& &L L p t ⎞() = 2πcos x() p x() t Kp z()⎝⎜⎠⎟ − t(1)There are two terms in Equation 1 that determine therate of change of force magnitude. The term on the right,Kṗ z(t), responds to stylus velocity in the normal directionto the underlying plane ṗ z(t) with a gain of K. Theterm on the left is due to the virtual textures. Here, thelateral velocity of the stylus ṗ x(t) is amplified with threeconstant gains (K, A, and 1/L) and one variable gain thatdepends on the lateral position of the stylus p x(t).Increasing A or decreasing L results in a faster changein force magnitude which can cause a textured surfaceto be perceived as less stable, or equivalently, result in asmaller stiffness threshold K T.We expected that d 2(t) would generate perceptuallymore stable textures than d 1(t) because it removed discontinuitiesin force commands at the texture entrypoints. This expectation was confirmed, except for thecondition where the subjects freely explored the virtualhaptic textures rendered with F vec(t). The stiffnessthresholds measured using F vec(t) and free explorationwere practically zero for both d 1(t) and d 2(t), and hencedid not exhibit any significant trend. The reason that thetextures felt unstable was the presence of strong buzzingnoises whenever we positioned the Phantom stylus deepinside the textured surfaces.Our finding that textures rendered with F mag(t) resultedin larger stiffness thresholds than those rendered withF vec(t) was also consistent with the nature of these tworendering methods. While F mag(t) imposed perturbationsin the force magnitude only, F vec(t) resulted in perturbationsin the force direction as well as force magnitude.The sometimes abrupt changes in force direction couldcause virtual textures rendered with F vec(t) to be perceivedas less stable than those rendered with F mag(t).Perceptually, F vec(t) is a useful rendering method becauseit can produce textures that feel much rougher than thoserendered with F mag(t) using the same texture model.We expected the experimentally confirmed fact thatstroking would result in a larger stiffness threshold thanfree exploration for the same rendering parameters. Oursubjects rarely used stroking in the free explorationmode although it was allowed. Instead, they chose toposition the stylus at various locations on or inside thevirtual textured surface to focus on the detection of perceivedinstability. Therefore, in the free explorationmode, the subjects concentrated on the detection ofunrealistic vibrations in the absence of any other signals.In the stroking mode, the subjects always felt thevibrations due to the stylus stroking the virtual texturedsurface. They had to detect additional noise to declarethe textured surface as unstable. Due to possible maskingof the different vibrations coming from the texturedsurface, it’s conceivable that subjects could not detectinstability with stroking as easily as they would with staticpositioning of the stylus. Indeed, our subjects reportedthat the experiments with stroking were moredifficult to perform.Frequently observed perceivedinstabilitiesWe found three types of frequently reported perceivedinstability in the psychophysical experiments: buzzing,aliveness, and ridge instability. The first two relate tothe perception of force magnitude, while the otherrelates to the perception of force direction. Buzzingrefers to high-frequency vibrations that subjects felt atthe Phantom stylus when it touched virtual textured surfaces.We observed this type of perceived instability inmost experimental conditions, particularly when thestiffness values were much higher than the thresholdsmeasured in the psychophysical experiments. The subjectsreported that buzzing appears to be of higher frequenciesthan the vibrations induced by the stroking ofvirtual textures.Measurement data supported the anecdotal report.Whenever the subjects felt buzzing, spectral componentsin a high-frequency region (roughly 150 to 250 Hz)appeared in the power spectral densities of the vibrationstransmitted through the stylus. An example is shown inFigure 5. In this figure, the horizontal axis represents frequencyfrom 10 to 500 Hz, while the vertical axis showsthe power spectrum density of p z(t) (the measured stylusposition along the normal direction of the texturedplane) in dB relative to 1-micrometer peak sinusoidalmotion. We can observe a spectral peak at around 71 Hz.Additional prominent spectral peaks appear in the high44 March/April 2004A129

frequency region starting at around150 Hz. The intensities of the highfrequencyvibrations are up to 25 dBabove the human detection thresholdsat the corresponding frequency(the red dotted line). These high-frequencyspectral peaks caused thebuzzing.We suspected that the rapidlychanging force commands for texturerendering might have excitedthe high-frequency dynamics of thePhantom, thereby causing high-frequencyvibration to be transmittedthrough the stylus. We thereforemeasured the frequency responseof the Phantom near the origin of itsworld coordinate frame and foundthat the Phantom indeed exhibiteda mechanical resonance at 218 Hz.This resonance was likely the sourceof the high-frequency spectralpeaks that invoked the perceptionof buzzing.The second type of instability thatthe subjects frequently observedwas aliveness. It occurred when thePhantom stylus was apparently heldstill yet the subject felt pulsatingforce changes emanating from thetextured surface. The sensationappeared to be at a lower frequencythan that of buzzing. Aliveness wasreported for textures rendered withF mag(t) (fixed force-direction) usingd 2(t) as penetration depth (continuouslyvarying force commands).The measured physical characteristicsof perceived aliveness were differentfrom those of buzzing.|P z (f )|(dB relative to 1-µm peak)Analyses in the frequency domain shed little insight onthe signals responsible for the percept of aliveness. However,examination of data in the time domain revealedmany instances where perceptible changes in forceoccurred while the stylus was perceived to be stationaryin space along the direction of the force changes.In Figure 6, the two horizontal axes indicate positionnormal to the textured surface p z(t) and along the lateralstroking direction p x(t). The vertical axis shows forcesfelt by the subject’s hand. The duration of the data set is400 ms. The large change in p x(t) was the result of thesubject stroking the textured surface. In contrast, therewas little change in p z(t). The change in force was on theorder of 0.5 Newtons. As a result, the subject felt a noticeablechange in normal force although the stylus was perceivedto be barely moving into the textured surface.Therefore, the force variation was interpreted as comingfrom an alive textured surface. Indeed, subjects sometimesreferred to the virtual object as a pulsating texturedsurface. These observations suggest that aliveness wascaused by larger-than-expected force variations in spite ofposition changes that were barely perceivable.6040200Power spectral density of P z (t)−2010 1 f tex 10 2 f insFrequency (Hz)5 Frequency domain analysis of the signals responsible for buzzing.F zS (t) (N)0.50.250−0.25−0.520S (px (t), F (t)) z10p z (t) (mm)0−10− 20S (px (t), p z (t), F (t)) z(p x (t), p z (t))−20−10Human detection thresholdsWe suspected that, unlike buzzing, which was causedby unstable control of haptic interface, aliveness wasprobably caused by inaccurate environment dynamics.To investigate this hypothesis, we examined whether itwas possible for a user to perceive aliveness while thetexture-rendering system including the force-feedbackdevice was stable in the control sense. We applied a passivity-basedcontrol theory to data measured from a userinteracting with virtual textured surfaces. You canregard a dynamic system as passive if it preserves or dissipatesits initial energy despite its interaction with anexternal system. Because passivity is a sufficient conditionfor control stability, 8 our hypothesis could be confirmedif we found cases in which a user perceivedaliveness from a passive texture rendering system.Using a passivity observer—an online observer formonitoring the energy flow of a dynamic system—weconfirmed that aliveness perception could indeed occurwhen the haptic texture-rendering system was passiveand stable. An example is shown in Figure 7. In this figure,the top panel shows the position data along the lateralstroking direction p x(t), the second panel shows the0S (pz (t), F (t)) zp x (t) (mm)6 Analysis of aliveness perception in the time domain. F s z(t) denotes the force measured alongthe long axis of the styles.1020A130IEEE Computer Graphics and Applications 45

Haptic Rendering—Beyond Visual ComputingPassivity observer W(N mm/sec) F (t) (N) z p z (t) (mm) p x (t) (mm)1000−1000 2 4 6 8 1050−50 2 4 6 8 1050−50 2 4 6 8 102000−2000 2 4 6 8 10Time (sec.)7 Example of using a passivity observer to analyze data corresponding to aliveness perception.F w z denotes the force measured along the direction normal to the textured surface.Stylus orientationzyxStylus tipp(t)n T (p(t))User-applied forcePhantom response forceResulting force8 Illustration of the force components involved inridge instability.position variable in the normal direction p z(t), the thirdpanel shows the force along the normal direction F z W (t),and the bottom panel shows the values of the passivityobserver. We can see that despite the abrupt forcechanges that resulted in the perception of aliveness, thepassivity observer remained positive. These results provideunequivocal evidence that perceived instability canoccur even when a haptic texture-rendering system ispassive and stable. We have therefore shown indirectlythat environment modeling and human perception canalso play important roles in perceived quality of a haptictexture-rendering system.Consider the difference between touching a real and avirtual surface. When a stylus touches a real surface, it’seither on or off the surface, but not inside the surface.When a stylus touches a virtual surface, however, the stylusmust penetrate the virtual surface for the user to forma perception of that surface through the resultant forcevariations. With a real surface, a stylus resting on the surfacecan remain stationary. With a virtual surface, however,the stylus’s position can fluctuate inside the surfaceand this fluctuation is amplified to result in perceivableforce variations by a texture renderer,thereby contributing to the perceptionof aliveness. It is well knownthat humans tend to rely more onvision for position-movement information,and that we can easily integratevisual position informationwith haptic force information. Ourrelatively poor kinesthetic resolutionof unsupported hand movements infree space—combined with our relativelyhigh sensitivity to forcechanges—is also responsible for theperception of aliveness.The last type of perceived instability,called ridge instability, is differentfrom the first two types in thesense that it is related to the perceptionof force directions. We use theterm ridge instability to refer to thephenomenon that the Phantom styluswas actively pushed to the valleysof the virtual textures rendered withF vec(t) when the stylus was placed onthe ridges of the textures. When areal stylus rests on the ridge of a real surface with sinusoidalgratings, the reaction force and friction of the surfacecombine to counterbalance the force exerted by theuser’s hand holding the stylus, thereby creating an equilibrium.The force rendered by F vec(t), however, was solelybased on the local texture geometry and did not takeinto account the direction of user-applied force, as illustratedin Figure 8. In this figure, we assume that the forceapplied by the user was normal to the plane underneaththe texture. According to the environment model F vec(t),the force applied by the Phantom was always in the directionof the surface normal n T(p(t)). As a result, the netforce exerted on the tip of the stylus—the sum of theforces applied by the user and the Phantom—was directedtoward the valley of the sinusoidal grating. Therefore,the subject who tried to rest the stylus on the ridge couldfeel the stylus being actively pushed into the valley.ConclusionsIn this article, we have shown that current haptic texture-renderingsystems might suffer from several typesof perceived instability. We also have demonstrated thatperceived instability can come from many sources,including the traditional control instability of hapticinterfaces as well as inaccurate modeling of environmentdynamics and the difference in sensitivity to forceand position changes of the human somatosensory system.Our work underscores the importance of developingtexture-rendering algorithms that guarantee theperceptual realism of virtual haptic textures. It is ourhope that this article will encourage more researchers tocontribute to the study of perceived instability of virtualhaptic textures.■AcknowledgmentsThis work was supported in part by a National ScienceFoundation (NSF) Faculty Early Career Develop-46 March/April 2004A131

ment (CAREER) Award under grant 9984991-IIS and inpart by an NSF award under grant 0098443-IIS. Wethank Blake Hannaford and Jee-Hwan Ryu for discussionson the passivity observer. We also thank VincentHayward for discussions on velocity estimation. Thethoughtful comments from the anonymous reviewersare greatly appreciated.References1. T.H. Massie, Initial Haptic Explorations with the Phantom:Virtual Touch through Point Interaction, master’s thesis,Dept. of Mechanical Eng., Massachusetts Inst. of Technology,1996.2. C. Ho, C. Basdogan, and M.A. Srinivasan, “Efficient Point-Based Rendering Techniques for Haptic Display of VirtualObjects,” Presence, vol. 8, no. 5, 1999, pp. 477-491.3. S. Choi and H.Z. Tan, “Perceived Instability of Virtual HapticTexture. I. Experimental Studies,” Presence, 2003, to bepublished.4. S. Choi and H.Z. Tan, “An Analysis of Perceptual InstabilityDuring Haptic Texture Rendering,” Proc. 10th Int’l Symp.Haptic Interfaces for Virtual Environment and TeleoperatorSystems (IEEE VR 02), IEEE CS Press, 2002, pp. 129-36.5. S. Choi and H.Z. Tan, “A Study on the Sources of PerceptualInstability During Haptic Texture Rendering,” Proc. IEEEInt’l Conf. Robotics and Automation, IEEE CS Press, 2002,pp. 1261-1268.6. S. Choi and H.Z. Tan, “An Experimental Study of PerceivedInstability During Haptic Texture Rendering: Effects of CollisionDetection Algorithm,” Proc. 11th Int’l Symp. HapticInterfaces for Virtual Environment and Teleoperator Systems(IEEE VR 03), IEEE CS Press, 2003, pp. 197-204.7. S. Choi and H.Z. Tan, “Aliveness: Perceived Instability froma Passive Haptic Texture Rendering System,” Proc.IEEE/RSJ Int’l Conf. Intelligent Robots and Systems, IEEE CSPress, 2003, pp. 2678-2683.8. B. Hannaford and J.-H. Ryu, “Time-Domain Passivity Controlof Haptic Interfaces,” IEEE Trans. Robotics and Automation,IEEE CS Press, vol. 18, no. 1, 2002, pp. 1-10.9. S.J. Lederman and R.L. Klatzky, “Hand Movement: A Windowinto Haptic Object Recognition,” Cognitive Psychology,vol. 19, 1987, pp. 342-368.10. G.A. Gescheider, Psychophysics: Method, Theory, and Application,2nd ed., Lawrence Erlbaum Assoc., 1985.Seungmoon Choi is a postdoctoralresearch associate at the HapticInterface Research Laboratory atPurdue University. His researchinterests include haptic rendering,psychophysics, and robotics. Choireceived a PhD in electrical and computerengineering from Purdue University.Hong Z. Tan is an associate professorin the school of electrical andcomputer engineering and the schoolof mechanical engineering (courtesy)at Purdue University. Herresearch interests include hapticinterfaces, psychophysics, haptic rendering,and distributed contact sensing. Tan received a PhDin electrical engineering and computer science from theMassachusetts Institute of Technology.Readers may contact Hong Tan at Purdue Univ., ElectricalEng. Building, 465 Northwestern Ave., WestLafayette, IN 47907; hongtan@purdue.edu.IEEE Transactions onMobile ComputingArevolutionary new quarterly journal that seeks out and delivers the verybest peer-reviewed research results on mobility of users, systems, data,computing information organization and access, services, management,and applications. IEEE Transactions on Mobile Computing gives youremarkable breadth and depth of coverage …ArchitecturesSupport ServicesAlgorithm/Protocol Design and AnalysisMobile EnvironmentMobile Communication SystemsApplicationsEmerging TechnologiesSubscribeNOW!To subscribe:http://computer.org/tmcor callUSA and CANADA:+1 800 678 4333WORLDWIDE:+1 732 981 0060A132IEEE Computer Graphics and Applications 47

Haptic Display of Interaction between Textured ModelsMiguel A. Otaduy Nitin Jain Avneesh Sud Ming C. LinDepartment of Computer ScienceUniversity of North Carolina at Chapel Hillhttp://gamma.cs.unc.edu/HTexturesFigure 1: Haptic Display of Interaction between Textured Models. From left to right: (a) high-resolution textured hammer (433K polygons) andCAD part (658K polygons), (b) low-resolution models (518 & 720 polygons), (c) hammer texture with fine geometric detail.ABSTRACTSurface texture is among the most salient haptic characteristics ofobjects; it can induce vibratory contact forces that lead to perceptionof roughness. In this paper, we present a new algorithm todisplay haptic texture information resulting from the interaction betweentwo textured objects. We compute contact forces and torquesusing low-resolution geometric representations along with textureimages that encode surface details. We also introduce a novel forcemodel based on directional penetration depth and describe an efficientimplementation on programmable graphics hardware that enablesinteractive haptic texture rendering of complex models. Ourforce model takes into account important factors identified by psychophysicsstudies and is able to haptically display interaction dueto fine surface textures that previous algorithms do not capture.Keywords: haptics, textures, graphics hardware1 INTRODUCTIONHaptic rendering provides a unique, two-way communication betweenhumans and interactive systems, enabling bi-directional interactionvia tactile sensory cues. By harnessing the sense of touch,haptic display can further enhance a user’s experience in a multimodalsynthetic environment, providing a more natural and intuitiveinterface with the virtual world. A key area in haptics that hasreceived increasing attention is the rendering of surface texture, i.e.fine geometric features on an object’s surface. The intrinsic surfaceproperty of texture is among the most salient haptic characteristicsof objects. It can be a compelling cue to object identity, and it canstrongly influence forces during manipulation [16]. In medical applicationswith limited visual feedback, such as minimally-invasiveor endoscopic surgery [24], and virtual prototyping applications ofmechanical assembly and maintainability assessment [27], accuratehaptic feedback of surface detail is a key factor for successful meticulousoperations.Most of the existing haptic rendering algorithms have focusedprimarily on force rendering of rigid or deformable flat polygonalmodels. This paper addresses the simulation of forces andtorques due to interaction between two textured objects. Effectivephysically-based force models have been proposed to render theinteraction between the tip (a point) of a haptic probe and a texturedobject [18, 10]. However, no technique is known to displayboth interaction forces and torques between two textured models.In fact, computation of texture-induced forces using full-resolutiongeometric representations of the objects and handling contacts atmicro-geometric scale is computationally prohibitive.Similar to graphical texture rendering [2], objects with high combinatorialcomplexity (i.e. with a high polygon count) can be describedby coarse representations with their fine geometric detailstored in texture images, which we will refer to as haptic texturesin this paper. Given this representation and a new force model thatcaptures the effect of geometric surface details, we are able to hapticallydisplay intricate interaction between highly complex modelsusing haptic textures instead of actual surface geometry.Main Contributions: In this paper, we introduce a physicallybasedalgorithm for incorporating texture effects to haptic displayof interaction between two polygonal models. This algorithm enables,for the first time, interactive haptic display of forces andtorques due to fine surface details. The main results of our paperare:• A novel force model for haptic texture rendering, based onthe gradient of directional penetration depth, that accounts forimportant factors identified by psychophysics studies;• A fast algorithm for approximating directional penetrationdepth between textured objects;• An efficient implementation on programmable graphics hardwarethat enables interactive haptic display of forces andtorques between complex textured models;A133

• A new approach to haptically render complex interaction dueto fine surface details using simplified representations of theoriginal models and the corresponding haptic textures.Our algorithm can be integrated in state-of-the-art haptic renderingalgorithms to enhance the range of displayed stimuli. Wehave successfully tested and demonstrated our algorithm and implementationon several complex textured models. Some examplesare shown in Fig. 1. Subjects were able to perceive roughness ofvarious surface textures.Organization: The rest of the paper is organized as follows. InSec. 2 we discuss related work. Sec. 3 defines key terminology anddescribes several important concepts central to our force model.Sec. 4 presents the force computation model. Sec. 5 introduces asimple yet effective algorithm for approximating directional penetrationdepth and its parallel implementation on graphics processors.We then describe our results in Sec. 6. Finally, we discuss andanalyze our approach in Sec. 7 and conclude with possible futureresearch directions in Sec. 8.2 PREVIOUS WORKIn this section we briefly discuss related work on haptic renderingand penetration depth computations.2.1 Six Degree-of-Freedom HapticsHaptic display of forces and torques between two interacting objectsis commonly known as 6 degree-of-freedom (DoF) haptics.In all approaches to 6-DoF haptics, collision detection is a dominantcomputational cost. The performance of collision detectionalgorithms depends on the size of the input models, which in turndepends on the sampling density of the models, both for polygonalrepresentations [23, 15, 11] and for voxel-based representations[17, 27].To be correctly represented, surfaces with high-frequency geometrictexture detail require higher sampling densities, thereby increasingthe cost of collision detection. As a result, haptic renderingof forces between textured objects becomes computationally infeasibleto achieve, and new representations must be considered.Otaduy and Lin [20] recently suggested multiresolution representationsto minimize the computational impact of collision detectionand to adaptively select the appropriate resolution at eachcontact location. However, their approach filters out high resolutiongeometric features, thus ignoring all texture effects.2.2 Haptic Texture RenderingRendering and perception of textures has been one of the most activeareas in haptics research. Please refer to [16] for a survey onpsychophysics of tactile texture perception. Klatzky and Ledermanmade important distinctions between perception of textures withbare skin vs. perception through a rigid object. When perceivedthrough a rigid probe, roughness of a textured surface is encoded asvibration.Several researchers have successfully developed haptic texturerendering techniques for interaction between a probe point and anobject, using coarse geometric approximations and geometric textureimages. These techniques use the idea of computing geometrydependenthigh frequency forces, which transmit vibratory informationto the user, and are perceived as virtual roughness. Minsky[18] showed that texture information can be conveyed by displayingforces on the tangent plane defined by the contact normal. Minskycomputed a texture-induced force proportional to the gradient of a2D height field stored in a texture map. Ho et al. [10] have proposedtechniques that alter the magnitude and direction of 3D normalforce based on height field gradient. Siira and Pai [26] followeda stochastic approach, where texture forces are computed accordingto a Gaussian distribution.All these techniques exploit the fact that, for point-object contact,a pair of texture coordinates can be well defined, and this isused to query height fields stored in texture maps. Note that onlygeometric effects of one object are captured. We are interested inrendering forces occurring during the interaction of two surfaces. Inthis case, the geometric interaction is not limited to and cannot bedescribed by a pair of contact points. Moreover, the local kinematicsof the contact between two surfaces include rotational degreesof freedom, not captured by point-based haptic rendering methods.Choi and Tan [3] have studied the influence of collision detectionand penetration depth computation on point-based haptic rendering,and their findings appear to be applicable to 6-DoF haptics as well.2.3 Penetration Depth ComputationSeveral algorithms [12, 6, 5, 13] have been proposed for computinga measure of penetration depth using various definitions. However,each of them assumes that at least one of the input models is aconvex polytope. It is commonly known that if two polytopes intersect,then the difference of their reference vectors with respect tothe origin of the world coordinate system lies in their convolutionor Minkowski sum [8]. The problem of penetration depth computationreduces to calculating the minimum distance from the origin tothe boundary of the Minkowski sum of two polyhedra. The worstcase complexity for two general, non-convex polyhedra can be ashigh as O(m 3 n 3 ), where m,n are the number of polygons in eachmodel. Kim et al. [14] presented an algorithm for estimating penetrationdepth between two polyhedral models using rasterizationhardware and hierarchical refinement. Although it offers better performancethan previous techniques, this approach may take up tominutes to compute the penetration depth, making it inadequate forhaptic simulation.In this paper we present a new algorithm to estimate directionalpenetration depth between models described by low-resolution representationsand haptic textures. Unlike the algorithm by Kim et al.[14], it does not compute the global penetration depth between twomodels, but its performance makes it suitable for haptic display.3 PRELIMINARIESIn this section we first introduce notation used in the paper. Then,we present definitions related to penetration depth, which is an essentialelement of our force model. Finally, we describe the computationalpipeline for haptic rendering of interaction between texturedmodels.3.1 NotationsA height field H is defined as a set H = {(x,y,z) | z =h(x,y),(x,y,z) ∈ R 3 }. We call h : R 2 → R a height function. Letq denote a point in R 3 , let q xyz = (q x q y q z ) T denote the coordinatesof q in a global reference system, and q uvn = (q u q v q n ) Tits coordinates in a rotated reference system {u,v,n}. A surfacepatch S ⊂ R 3 can be represented as a height field along a directionn if q n = h(q u ,q v ),∀q ∈ S. Then, we can define a mappingg : D → S,D ⊂ R 2 , as g(q u ,q v ) = q xyz , where:h(q u ,q v ) = q n = n · q xyz = n · g(q u ,q v ) (1)The inverse of the mapping g is the orthographic projection of Sonto the plane (u,v) along the direction n.A134

3.2 Definitions of Penetration DepthPenetration depth δ between two intersecting polytopes is typicallydefined as the minimum translational distance required to separatethem (see Fig. 2-b). As mentioned in Sec. 2.3, this distance is equivalentto the distance from the origin to the Minkowski sum of thepolyhedra. Directional penetration depth δ n along the direction n isdefined as the minimum translation along n to separate the polyhedra(see Fig. 2-c). The penetration depth between two intersectingsurface patches will be referred to as local penetration depth.1. Each haptic simulation frame starts by performing collisiondetection between the low-resolution meshes. We then identifyintersecting surface patches as contacts. We characterizeeach contact by a pair of contact points on the patches and apenetration direction n.2. For each contact, we compute force and torque using ournovel force model for texture rendering, based on the penetrationdepth and its gradient. The penetration depth is approximatedtaking into account fine geometric detail stored inhaptic textures.3. The forces and torques of all contacts are combined to computethe net force and torque on the probe object.Other effects, such as friction [9], can easily be incorporated intothis display pipeline using the contact information computed betweenthe low-resolution meshes.4 A FORCE MODEL FOR TEXTURE RENDERINGFigure 2: Definitions of Penetration Depth. (a) Intersecting objects Aand B, (b) global penetration depth δ, and (c) directional penetrationdepth δ n along n.Let us assume that two intersecting surface patches S A and S Bcan be represented as height fields along a direction n. Consequently,S A and S B can be parameterized by orthographic projectionalong n, as expressed in Sec. 3.1. As a result of the parameterization,we obtain mappings g A : D A → S A and g B : D B → S B , as wellas height functions h A : D A → R and h B : D B → R. The directionalpenetration depth δ n of the surface patches S A and S B is the maximumheight difference along the direction n, as illustrated in Fig. 3by a 2D example. Therefore, we can define the directional penetrationdepth δ n as:(δ n = max hA (u,v) − h B (u,v) ) (2)(u,v)∈(D A ∩D B )Figure 3: Penetration Depth of Height Fields. Directional penetrationdepth of surface patches expressed as height difference.3.3 Haptic Display PipelineWe assume that the interacting objects can be described as parameterizedlow-resolution triangle meshes with texture maps that storefine geometric detail. In a haptic simulation of object-object interaction,the object whose motion is controlled by the user is calledthe probe object. Contacts between the probe object and the rest ofthe objects in the environment generate forces that are displayed tothe user.Following a common approach in 6-DoF haptics, we simulatethe dynamics of the probe object as a result of contact forces and avirtual coupling force that ensures stable interaction with the user[1]. We propose a novel algorithm for computing contact forces,taking into account texture effects. We follow the steps below tocompute contact forces:In this section we describe our force model for haptic display ofinteraction between textured surfaces. We first show how factorshighlighted by psychophysics studies are taken into account. Then,we introduce a penalty-based force model for texture rendering.Finally, we present the formulation of the gradient of penetrationdepth used in our force model.4.1 Foundation of the Proposed Force ModelRoughness of surface textures perceived through a rigid probe ismainly encoded as vibration and strongly influences the forces thatmust be applied to manipulate the objects [16]. In point-based haptictexture rendering, vibrating forces are commonly computed usinga height field gradient [18, 10]. Our force model generalizes thepoint-based approach by computing forces based on the gradient ofpenetration depth between two objects.Based on psychophysics studies, Klatzky and Lederman [16]highlight factors influencing perception of roughness through arigid spherical probe. These factors are:Probe Radius: For spherical probes, the texture frequency at whichperception of roughness is maximum depends on probe radius. Atlow frequencies, roughness increases with texture frequency, butafter reaching a peak, roughness decreases as texture frequency increases.Our conjecture is that roughness perception is tightly coupledto the trajectory traced by the probe, which can be regarded asan offset surface of the perceived geometry. Okamura and Cutkosky[19] also modeled interaction between robotic fingers and texturedsurfaces by tracing offset surfaces. They defined an offset surfaceas the boundary of the Minkowski sum of a given surface and asphere. Therefore, the height of the offset surface at a particularpoint is the distance to the boundary of the Minkowski sum for aparticular position of the probe, also known to be the penetrationdepth 1 . In other words, the height of the offset surface reflects thedistance that the probe must move in order to avoid interpenetrationwith the surface. Since, for spherical probes, perception ofroughness seems to be tightly coupled with the oscillation of offsetsurfaces, in our force model for general surfaces we have taken intoaccount the variation of penetration depth, i.e. its gradient.Normal Force: Perception of roughness grows monotonically withnormal force. This relation is also captured by our force model in1 Actually, the height of the offset surface is the distance to the surfacealong a particular direction, so the distance to the boundary of theMinkowski sum must also be measured along a particular direction. Thisis known to be the directional penetration depth.A135

a qualitative way, in making tangential forces and torques proportionalto the normal force.Exploratory Speed: The exploratory speed, or velocity of theprobe in the plane of contact with the surface, affects the perceptionof roughness. Our force model is intrinsically geometry-based, butin a haptic simulation dynamic effects are introduced by the hapticdevice and the user. We have analyzed the dynamic behavior of ourforce model, and we have observed that vibratory motion producedby simulated forces behaves in a way similar to physical roughnessperception. The results of our experiments are described in detailin [21].The influence of probe geometry, normal force and exploratoryspeed is taken into consideration in the design of our force model,which will be presented next.4.2 Penalty-Based Texture ForceFor two objects A and B in contact, we define a penalty-based forceproportional to the penetration depth δ between them. Penaltybasedforces are conservative, and they define an elastic potentialfield. In our force model we have extended this principle to computetexture-induced forces between two objects.We define an elastic penetration energy U with stiffness k as:U = 1 2 kδ 2 (3)Based on this energy, we define force F and torque T as:(FT)= −∇U = −kδ (∇δ) (4)( )where ∇ = ∂∂x , ∂ ∂y , ∂ ∂z , ∂ ∂ ∂∂θx , ∂θy , ∂θz is the gradient in 6-DoF configurationspace.As described in Sec. 3.3, each contact between objects A and Bcan be described by a pair of contact points p A and p B , and by apenetration direction n. We assume that, locally, the penetrationdepth between objects A and B can be approximated by the directionalpenetration depth δ n along n. We rewrite Eq. 4 for δ n in areference system {u,v,n} 2 . In this case, Eq. 4 reduces to:( Fu F v F n T u T v T n) T = −kδn( ∂δn∂u∂δ n∂v1 ∂δ n∂θ u∂δ n∂θ v∂δ n∂θ n) T(5)where θ u , θ v and θ n are the rotation angles around the axes u, v andn respectively.The force and torque on object A (and similarly on object B) foreach contact can be expressed in the global reference system as:4.3 Penetration Depth and GradientIn our formulation, δ and δ n are functions defined on a 6-DoF configurationspace. We have opted for central differencing over onesideddifferencing to approximate ∇δ n , because it offers better interpolationproperties and higher order approximation. The partialderivatives are computed as:∂δ n∂u = δ n(u+∆u,v,n,θ u ,θ v ,θ n ) − δ n (u − ∆u,v,n,θ u ,θ v ,θ n )2∆uand similarly for ∂δ n∂v , ∂δ n, ∂δ nand ∂δ n.∂θ u ∂θ v ∂θ nδ n (u + ∆u,...) can be obtained by translating object A a distance∆u along the u axis and computing the directional penetrationdepth. A similar procedure is followed for other penetration depthvalues.5 DIRECTIONAL PENETRATION DEPTHIn this section we present an algorithm for approximating local directionalpenetration depth for textured models and describe a parallelimplementation on graphics hardware.5.1 Approximate Directional Penetration Depth between TexturedModelsA contact between objects A and B is defined by two intersectingsurface patches S A and S B . The surface patch S A is approximated bya low-resolution surface patch Ŝ A (and similarly for S B ). We definef A : Ŝ A → S A , a mapping function from the low-resolution surfacepatch Ŝ A to the surface patch S A .Collision detection between two low-resolution surfaces patchesŜ A and Ŝ B returns a penetration direction n. Let us assume that bothS A and Ŝ A (and similarly for S B and Ŝ B ) can be represented as heightfields along n, following the definition in Sec. 3.1. Given a rotatedreference system {u,v,n}, we can project S A and Ŝ A orthographicallyalong n onto the plane (u,v). As the result of this projection,we obtain mappings g A : D A → S A and ĝ A : ˆD A → Ŝ A . We define¯D A = D A ∩ ˆD A .The mapping function g A can be approximated by a compositemapping function f A ◦ ĝ A : ¯D A → S A (See Fig. 4). From Eq. 1, wedefine an approximate height function ĥ : ¯D A → R as:(7)ĥ(u,v) = n ·(f A ◦ ĝ A (u,v)) (8)F A = (u v n)(F u F v F n ) TT A = (u v n)(T u T v T n ) T (6)As explained in Sec. 3.3, forces and torques of all contacts aresummed up to compute the net force and torque.Generalizing Minsky’s approach [18], we define tangentialforces F u and F v proportional to the gradient of penetration depth.However, we also define a penalty-based normal force and gradientdependenttorque that describe full 3D object-object interaction.In addition, in our model the tangential force and the torque areproportional to the normal force, which is consistent with psychophysicsstudies showing that perceived roughness increases withthe magnitude of the normal force [16].2 u and v may be selected arbitrarily as long as they form an orthonormalbasis with n.Figure 4: Approximate Height Function. Height function of a surfacepatch approximated by a composite mapping function.Given approximate height functions ĥ A and ĥ B , a domain D =¯D A ∩ ¯D B , and Eq. 2, we can approximate the directional penetrationdepth δ n of S A and S B by:δˆn = max(ĥA (u,v) − ĥ B (u,v) ) (9)(u,v)∈DAlthough this algorithm can be realized on CPUs, it is bestsuited for implementation on graphics processors (GPUs), as wewill present next.A136

one of the RGBA channels. This optimization allows us to exploit5.2 Computation on Graphics Hardware5 http://www.nvidia.com/dev content/nvopenglspecs/GL NV occlusion query.txt chronously.vector computation capabilities of fragment processors. As shownAs shown in Eq. 5, computation of 3D texture-induced force andin Fig. 5, we also tile 11 height differences per contact in the depthtorque according to our model requires the computation of directionalpenetration depth δ n and its gradient at every contact. Frombuffer.Eq. 7, this reduces to computing δ n all together at 11 configurationsof object A 3 . As pointed out in section 2.3, computation of penetrationdepth using exact object-space or configuration-space algorithmsis too expensive for haptic rendering applications. Instead,the approximation δ ˆ n according to Eqs. 8 and 9 leads to a naturaland efficient image-based implementation on programmable graphicshardware. The mappings ĝ and f correspond, respectively, toorthographic projection and texture mapping operations, which arebest suited for the parallel and grid-based nature of GPUs.For every contact, we first compute ĥ B , and then perform two Figure 5: Tiling in the GPU. Tiling of multiple height functions andoperations for each of the 11 object configurations: compute ĥcontacts to minimize context switches between target buffers.Afor the transformed object A, and then find the penetration depthδˆn = max(∆ĥ) = max ( )ĥ A − ĥ 4 Multiple Simultaneous Contacts:B .The computational cost of haptic texture rendering increases linearlywith the number of contact patches between the interactingHeight ComputationIn our GPU-based implementation, the mapping f : Ŝ → S is implementedobjects. However, performance can be further optimized. In orresolutionas a texture map that stores geometric detail of the highderto limit context switches, we tile the height functions associatedsurface patch S. We refer to f as a “haptic texture”. The with multiple pairs of contact patches in one single texture t, and wemapping ĝ is implemented by rendering Ŝ using an orthographic tile the height differences in the depth buffer, as shown in Fig. 5. Weprojection along n. We compute the height function ĥ in a fragment also minimize the cost of “max search” operations by performingprogram. We obtain a point in S by looking up the haptic texture f occlusion queries on all contacts in parallel.and then we project it onto n. The result is stored in a floating pointtexture t.6 RESULTSWe choose geometric texture mapping over other methods forapproximating h (e.g. rendering S directly or performing displacementmapping) in order to maximize performance. We store theinput haptic texture f as a floating point texture, thus alleviatingWe now describe the implementation details and results obtainedwith our haptic texture rendering algorithm, both in terms of forceand motion characteristics, as well as performance.precision problems.Max Search6.1 Implementation DetailsThe max function in Eq. 9 can be implemented as a combinationOur haptic texture rendering algorithm requires a preprocessingof frame buffer read-back and CPU-based search. However, westep. Input models are assumed to be 2-manifold triangle meshesavoid expensive read-backs by posing the max function as a binarywith fine geometric details. We parameterize the meshes and createtexture atlas storing surface positions. We also simplify thesearch on the GPU [7]. Given two height functions ĥ A and ĥ B storedin textures t 1 and t 2 , we compute their difference and store it inmeshes to produce coarse resolution approximations which are usedthe depth buffer. We scale and offset the height difference to fit inby the collision detection module. The parameterization must bethe depth range. Height subtraction and copy to depth buffer arepreserved during the simplification process, and distortion mustperformed in a fragment program, by rendering a quad that coversbe minimized. Our implementation of parameterization-preservingthe entire buffer. For a depth buffer with N bits of precision, thesearch domain is the integer interval [0,2 N simplification is based on existing approaches [25, 4].). The binary searchstarts by querying if there is any value larger than 2 N−1 As described in Sec. 3.3, before computing forces we perform. We rendera quad at depth 2 N−1 and perform an occlusion query 5 collision detection between coarse-resolution models. We adapt the, which will approach of Kim et al. [15] and decompose the models in convexreport if any pixel passed the depth test, i.e. the stored depth waslarger than 2 N−1 pieces. Object interpenetration is considered to occur when objects. Based on the result, we set the depth of a new are closer than a distance tolerance. In practice, by using this technique,penetration depth between the coarse resolution models isquad and continue the binary search.Gradient Computationcomputed less frequently, thus accelerating collision detection.The height functions ĥ A (±∆u), ĥ A (±∆v) and ĥ A (±∆θ n ) may be For texture force computation, we compute each value of penetrationdepth between contact patches on a 50 × 50, 16-bit depthobtained by simply translating or rotating ĥ A (0). As a result, only 6buffer. This resolution proved to be sufficient based on the results.height functions ĥ A (0), ĥ B (0), ĥ A (±∆θ u ) and ĥ A (±∆θ v ) need to beIn our experiments we have used a 6-DoF Phantomcomputed for each pair of contact patches. These 6 height functionshapticdevice, a dual Pentium-4 2.4GHz processor PC with 2.0are tiled in one single texture t to minimize context switches andGB of memory and an NVidia GeForce FX5950 graphics card,increase performance (See Fig. 5). Moreover, the domain of eachand Windows2000 OS. The penetration depth computation onheight function is split into 4 quarters, each of which is mapped tographics hardware is implemented using OpenGL plus OpenGL’s3 Note that, since we use central differencing to compute partial derivativesof δ n , we need to transform object A to two different configurations and Our haptic texture rendering cannot be stalled by the visual dis-ARB fragment program and GL NV occlusion query extensions.recompute δ n . All together we compute δ n itself and 5 partial derivatives, play of the scene; hence, it requires a dedicated graphics card.hence 11 configurationsWe display the full resolution scene on a separate commodity PC.4 We denote the height difference at the actual object configuration by∆ĥ(0), and the height differences at the transformed configurations by∆ĥ(±∆u), ∆ĥ(±∆v), ∆ĥ(±∆θ u ), ∆ĥ(±∆θ v ) and ∆ĥ(±∆θ n ).The force update of the haptic device takes place at a frequency of1kHz, but the haptic simulation is executed in a separate thread thatupdates the force input to a stabilizing virtual coupling [1] asyn-A137

Figure 6: Benchmark Models. From left to right: (a) textured blocks, (b) block and gear, (c) hammer and torus, (d) file and CAD part.6.2 BenchmarksIn our experiments we have used the models shown in Fig. 6. Thecomplexity of full resolution textured models and their coarse resolutionapproximations is listed in Table 1.Models Full Res. Tris Low Res. Tris Low Res. PcsBlock 65536 16 1Gear 25600 1600 1Hammer 433152 518 210CAD Part 658432 720 390File 285824 632 113Torus 128000 532 114Table 1: Complexity of Benchmark Models. Number of triangles atfull resolution (Full Res. Tris) and low resolution (Low Res. Tris),and number of convex pieces at low resolution (Low Res. Pcs).Notice the drastic simplification of the low resolution models.At this level all texture information is eliminated from the geometry,but it is stored in 1024 × 1024-size floating point textures. Thenumber of convex pieces at coarse resolution reflects the geometriccomplexity for the collision detection module. Also notice that theblock and gear models are fully convex at coarse resolution. The interactionbetween these models is described by one single contact,so they are better suited for analyzing force and motion characteristicsin the simulations.6.3 Conveyance of RoughnessWe have performed experiments to test the conveyance of roughnesswith our haptic texture rendering algorithm.Roughness under Translation: The gear and block models presentridges that interlock with each other. One of our experiments consistedof translating the block in the 3 Cartesian axes, while beingin contact with the fixed gear, as depicted in Fig. 6-b. Fig. 7 showsthe position of the block and the force exerted on it during 1500frames of interactive simulation (approx. 3 seconds).Notice that the force in the x direction, which is parallel to theridges, is almost 0. Our model successfully yields this expected result,because the derivative of the penetration depth is 0 along the xdirection. Notice also the staircase shape of the motion in the z direction,which reflects how the block rests for short periods of timeon the ridges of the gear. The motion in the y direction resembles astaircase as well, but with small overshoots. These reflect the statebetween two successive interlocking situations, when the ridges areopposing each other. The wide frequency spectrum of staircase motionis possible due to the fine spatial resolution of penetration depthand gradient computation. Last, the forces in y and z are correlatedwith the motion profiles.Roughness under Rotation: We placed two identical stripedblocks interlocking each other, as shown in Fig. 6-a. We thenperformed small rotations of the upper block around the directionn, and observed the induced translation along that same direction.Fig. 8 shows the rotation and translation captured during 600080706050403020100Position (mm.)xyz−100 500 1000 15000.90.80.70.60.50.40.30.20.10Forces (N)F xF yF z−0.10 500 1000 1500Simulation framesFigure 7: Roughness under Translation. Position and force profilesgenerated while translating the model of a textured block in contactwith a gear model, as shown in Fig. 6-b. Notice the staircase likemotion in z, and the correlation between force and position changes.frames of interactive haptic simulation (approx. 12 seconds). Noticehow the top block rises along n as soon as we rotate it slightly,thus producing a motion very similar to the one that occurs in reality.Point-based haptic rendering methods are unable to capture thistype of effect. Our force model successfully produces the desiredeffect by taking into account the local penetration depth betweenthe blocks. Also, the derivative of the penetration depth produces aphysically-based torque in the direction n that opposes the rotation.Summary of Experiments: From the experiments describedabove, we conclude that our force model successfully capturesroughness properties of objects with fine geometric detail. We havealso conducted informal experiments where subjects were asked toexplore a textured plate with a virtual probe, while only the untexturedcoarse-resolution models were displayed graphically on thescreen. Hence, the subjects could only recognize the texture patternsthrough haptic cues. The reported experiences were promising,as subjects were able to successfully describe regular patternssuch as stripes, but had more difficulty with irregular patterns. Thisresult is what we expect when real, physical textured models areexplored.6.4 Performance TestsOne of the key issues to achieve realistic haptic rendering is veryhigh force update rate. High update rates enhance the stability ofthe system, as well as the range of stimuli that can be displayed.We have tested the performance of our haptic texture rendering algorithmand its implementation in scenarios where the coarse resolutionmodels present complex contact configurations. These scenariosconsist of a file scraping a rough CAD part, and a texturedA138

6Motion along n (in mm.)4200 1000 2000 3000 4000 5000 600020−2Rotation around n (in deg.)0 1000 2000 3000 4000 5000 6000Simulation framesFigure 8: Roughness under Rotation. Motion profile obtained byrotating one textured block on top of another one, as depicted inFig. 6-a. Notice the translation induced by the interaction of ridgesduring the rotational motion.hammer touching a wrinkled torus. In particular, we show timingsfor 500 frames of the simulation of the file interacting with the CADpart in Fig. 9. The graph reflects the time spent on collision detectionbetween the coarse-resolution models (an average of 2ms), thetime spent on haptic texture rendering, and the total time per frame,which is approximately equal to the sum of the previous two. Inthis experiment we computed each value of penetration depth on a50 × 50 16-bit depth buffer (See Sec. 5.2). As shown in Sec. 6.3,this proved to be sufficient to display convincing roughness stimuli.1210864200 100 200 300 400 5005Time (in msec.)Total Frame TimeTexture ForcesCollision DetectionNumber of contact patches00 100 200 300 400 500Simulation framesFigure 9: Timings. Performance analysis and number of clusteredcontact patches during 500 simulation frames of a file model scrapinga CAD part, as shown in Fig. 6-d. In this complex contact scenario weare able to maintain a haptic frame rate between 100Hz and 200Hz.In this particularly challenging experiment we were able to obtainhaptic update rates between 100Hz and 200Hz. The dominantcost appears to be the haptic texture rendering, which dependsnearly linearly on the number of contacts. The achieved force updaterate may not be high enough to render textures with very highspatial frequency. However, as shown in Sec. 6.3, our proposedforce model enables perception of roughness stimuli that were notcaptured previously by earlier methods. Moreover, in Fig. 9 weshow performance results for a contact configuration in which largeareas of the file at many different locations are in close proximitywith the CAD part. In fact, collision detection using coarseresolutionmodels reports an average of 104 pairs of convex piecesin close proximity, which are later clustered into as many as 7 contacts.Using the full-resolution models, the number of contact pairsin close proximity would increase by several orders of magnitude,and simply handling collision detection would become challengingat the desired haptic rendering frame rates. Furthermore, asthe support for programming on GPUs and capabilities of GPUscontinue to grow at a rate faster than Moore’s Law, we expect theperformance of our algorithm to reach KHz update rates in the nearfuture.7 DISCUSSION AND ANALYSISIn Sec. 6.3 we have analyzed forces and motion generated by ouralgorithm during actual haptic simulations. We have further analyzedthe properties of the force model presented in Sec. 4 by simulatingits behavior in experiments similar to the ones conducted inpsychophysics studies [16]. Our main conclusion is that the accelerationproduced by our force model matches qualitatively the behaviorof roughness perception as a function of texture frequency.A detailed description of the experiments we have conducted canbe found in [21].Our force model and implementation present a few limitations,some of which are common to existing haptic rendering methods.Next we discuss these limitations.7.1 Force ModelIn some contact scenarios with large contact areas, the definitionof a local and directional penetration depth is not applicable. Anexample is the problem of screw insertion. Situations also exist inwhich local geometry cannot be represented as height fields, andthe gradient of directional penetration depth may not capture theeffect of interlocking features.As shown in Sec. 6, in practice our force model generates forcesthat create a realistic perception of roughness for object-object interaction;however, one essential limitation of penalty-based methodsand impedance-type haptic devices is the inability to enforcemotion constraints. Our force model attempts to do so by increasingtangential contact stiffness when the gradient of penetration depthis high. Implicit integration of the motion of the probe object allowsfor high stiffness and, therefore, small interpenetrations, but the perceivedstiffness of rigid contact is limited through virtual couplingfor stability reasons. New constraint-based haptic rendering techniquesand perhaps other haptic devices [22] will be required toproperly enforce constraints.A very important issue in every force model for haptic renderingis its stability. Choi and Tan [3] have shown that even passiveforce models may suffer from a problem called aliveness. In ouralgorithm, discontinuities in the collision detection between lowresolutionmodels are possible sources of aliveness.7.2 Frequency and Sampling IssuesAs with other sample-based techniques, our haptic texture renderingalgorithm is susceptible to aliasing problems. Here we discussdifferent aliasing sources and suggest some solutions.Input textures: The resolution of input textures must be highenough to capture the highest spatial frequency of input models,although input textures can be filtered as a preprocessing step todownsample and reduce their size.Image-based computation: In the height function computationstep, buffer resolution must be selected so as to capture the spatialA139

frequency of input models. Buffer size, however, has a significantimpact in the performance of force computation.Discrete derivatives: Penetration depth may not be a smooth function.This property results in an infinitely wide frequency spectrum,which introduces aliasing when sampled. Differentiation aggravatesthe problem, because it amplifies higher frequencies. The immediateconsequence in our texture rendering approach is that theinput texture frequencies have to be low enough so as to representfaithfully their derivatives. This limitation is common to existingpoint-based haptic rendering methods [18] as well.Temporal sampling. Force computation undergoes temporal samplingtoo. The Nyquist rate depends on object speed and spatialtexture frequency. Image-based filtering prior to computation ofpenetration depth may remove undesirable high frequencies, but itmay also remove low frequencies that would otherwise appear dueto the nonlinearity of the max search operation. In other words, filteringa texture with very high frequency may incorrectly removeall torque and tangential forces. Temporal supersampling appearsto be a solution to the problem, but is often infeasible due to thehigh update rates required by haptic simulation.8 CONCLUSIONWe have introduced a new haptic rendering algorithm for displayinginteraction between two textured models, based on localizeddirectional penetration depth and its gradient. We have also presentedan image-based implementation on programmable graphicshardware that enables interactive haptic display between complextextured models for the first time. We have further shown that, usinga coarse geometric representation with haptic textures that encodefine surface details, it is possible to render contact forces andtorques between two interacting textured models at haptic rates.Several possible directions for future research remain, includingbut not limited to:• Interactive haptic texture synthesis;• Addition of constraint forces for fine motion and dexterousmanipulation;• Further analysis of human factors.Finally, we would like to integrate our haptic rendering systemwith different applications, such as assisted technology, surgicaltraining, and virtual prototyping.ACKNOWLEDGMENTSThis research is supported in part by a fellowship of the Governmentof the Basque Country, National Science Foundation, Officeof Naval Research, U.S. Army Research Office, and Intel Corporation.We would like to thank Roberta Klatzky, Susan Lederman, DineshManocha, Russ Taylor, Fred Brooks, Mark Foskey, Bill Baxter,the UNC Gamma group and the anonymous reviewers for theirfeedback on the earlier drafts of this paper.REFERENCES[1] R. J. Adams and B. Hannaford. A two-port framework for the designof unconditionally stable haptic interfaces. Proc. of IEEE/RSJ InternationalConference on Intelligent Robots and Systems, 1998.[2] Edwin E. Catmull. A Subdivision Algorithm for Computer Displayof Curved Surfaces. PhD thesis, Dept. of CS, U. of Utah, December1974.[3] S. Choi and H. Z. Tan. Aliveness: Perceived instability from a passivehaptic texture rendering system. Proc. of IEEE/RSJ InternationalConference on Intelligent Robots and Systems, 2003.[4] J. Cohen, M. Olano, and D. Manocha. Appearance preserving simplification.In Proc. of ACM SIGGRAPH, pages 115–122, 1998.[5] D. Dobkin, J. Hershberger, D. Kirkpatrick, and S. Suri. Computingthe intersection-depth of polyhedra. Algorithmica, 9:518–533, 1993.[6] E.G. Gilbert and C.J. Ong. New distances for the separation and penetrationof objects. In Proceedings of International Conference onRobotics and Automation, pages 579–586, 1994.[7] N. K. Govindaraju, B. Lloyd, W. Wang, M. C. Lin, and D. Manocha.Fast computation of database operations using graphics processors.Proc. of ACM SIGMOD, 2004.[8] L. J. Guibas and J. Stolfi. Ruler, compass and computer: the designand analysis of geometric algorithms. In R. A. Earnshaw, editor, TheoreticalFoundations of Computer Graphics and CAD, volume 40 ofNATO ASI Series F, pages 111–165. Springer-Verlag, 1988.[9] V. Hayward and B Armstrong. A new computational model of frictionapplied to haptic rendering. Experimental Robotics VI, 2000.[10] C.-H. Ho, C. Basdogan, and M. A. Srinivasan. Efficient point-basedrendering techniques for haptic display of virtual objects. Presence,8(5):pp. 477–491, 1999.[11] D. Johnson and P. Willemsen. Six degree of freedom haptic renderingof complex polygonal models. In Proc. of Haptics Symposium, 2003.[12] S. S. Keerthi and K. Sridharan. Efficient algorithms for computing twomeasures of depth of collision between convex polygons. TechnicalReport, 1989.[13] Y. Kim, M. Lin, and D. Manocha. DEEP: an incremental algorithmfor penetration depth computation between convex polytopes. Proc. ofIEEE Conference on Robotics and Automation, pages 921–926, 2002.[14] Y. Kim, M. Otaduy, M. Lin, and D. Manocha. Fast penetration depthcomputation for physically-based animation. Proc. of ACM Symposiumon Computer Animation, 2002.[15] Y. J. Kim, M. A. Otaduy, M. C. Lin, and D. Manocha. Six-degreeof-freedomhaptic rendering using incremental and localized computations.Presence, 12(3):277–295, 2003.[16] R. L. Klatzky and S. J. Lederman. Perceiving texture through a probe.In M. L. McLaughlin, J. P. Hespanha, and G. S. Sukhatme, editors,Touch in Virtual Environments, chapter 10, pages 180–193. PrenticeHall PTR, Upper Saddle River, NJ, 2002.[17] W. McNeely, K. Puterbaugh, and J. Troy. Six degree-of-freedom hapticrendering using voxel sampling. Proc. of ACM SIGGRAPH, pages401–408, 1999.[18] M. Minsky. Computational Haptics: The Sandpaper System for SynthesizingTexture for a Force-Feedback Display. PhD thesis, Ph.D.Dissertation, Program in Media Arts and Sciences, MIT, 1995. Thesiswork done at UNC-CH Computer Science.[19] A. M. Okamura and M. R. Cutkosky. Feature detection for hapticexploration with robotic fingers. International Journal of RoboticsResearch, 20(12):925–938, 2001.[20] M. A. Otaduy and M. C. Lin. Sensation preserving simplificationfor haptic rendering. ACM Trans. on Graphics (Proc. of ACM SIG-GRAPH), pages 543–553, 2003.[21] M. A. Otaduy and M. C. Lin. A perceptually-inspired force model forhaptic texture rendering. Proc. of Symposium APGV, 2004.[22] M. Peshkin and J. E. Colgate. Cobots. Industrial Robot, 26(5):335–341, 1999.[23] D. Ruspini and O. Khatib. A framework for multi-contact multi-bodydynamic simulation and haptic display. Proc. of IEEE/RSJ InternationalConference on Intelligent Robots and Systems, 2000.[24] J. K. Salisbury. Making graphics physically tangible. Communicationsof the ACM, 42(8), 1999.[25] P. V. Sander, J. Snyder, S. J. Gortler, and H. Hoppe. Texture mappingprogressive meshes. Proc. of ACM SIGGRAPH, pages 409–416, 2001.[26] J. Siira and D. K. Pai. Haptic textures – a stochastic approach. Proc.of IEEE International Conference on Robotics and Automation, pages557–562, 1996.[27] M. Wan and W. A. McNeely. Quasi-static approximation for 6degrees-of-freedom haptic rendering. Proc. of IEEE Visualization,pages 257–262, 2003.A140

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.orgA Unified Treatment of Elastostatic Contact Simulationfor Real Time HapticsDoug L. James ∗ and Dinesh K. Pai ∗†University of British ColumbiaAbstractWe describe real-time, physically-based simulation algorithms forhaptic interaction with elastic objects. Simulation of contact withelastic objects has been a challenge, due to the complexity of physicallyaccurate simulation and the difficulty of constructing usefulapproximations suitable for real time interaction. We show that thischallenge can be effectively solved for many applications. In particularglobal deformation of linear elastostatic objects can be efficientlysolved with low run-time computational costs, using precomputedGreen’s functions and fast low-rank updates based onCapacitance Matrix Algorithms. The capacitance matrices constituteexact force response models, allowing contact forces tobe computed much faster than global deformation behavior. Vertexpressure masks are introduced to support the convenient abstractionof localized scale-specific point-like contact with an elasticand/or rigid surface approximated by a polyhedral mesh. Finally,we present several examples using the CyberGlove TM andPHANToM TM haptic interfaces.1 IntroductionDiscrete linear elastostatic models (LEMs) are importantphysically-based elastic primitives for computer haptics becausethey admit a very high-degree of precomputation, or “numericalcompression” [1], in a way that affords cheap force responsemodels suitable for force feedback rendering of stiff elastic objectsduring continuous contact. The degree of useful precomputationis quite limited for many types of nonlinear and/or dynamicalelastic models, but LEMs are an exception, mainly due to theprecomputability of time-independent Green’s functions (GFs)and the applicability of linear superposition principles. Intuitively,GFs form a basis for describing all possible deformations of aLEM. Thus, while LEMs form a relatively simple class of elasticmodels in which geometric and material linearities are an ultimatelimitation, the fact that the model is linear is also a crucial enablingfactor. We conjecture that LEMs will remain one of the bestruntime approximations of stiff elastic models for simulationsrequiring stable high-fidelity force feedback.A central idea for LEMs in computer haptics is the formulationof the boundary value problem (BVP) solution in terms of suitableprecomputed GFs using Capacitance Matrix Algorithms (CMAs).Derived from the Sherman-Morrison-Woodbury formula for lowrankupdating of matrix inverses (and factorizations), CMAs have along history in linear algebra [30, 16], where they have been commonlyused for static reanalysis [22], to efficiently solve LEM contactmechanics problems [12, 25] and more recently for interactiveLEM simulations [6, 21].For computer haptics, a fundamental reason for choosing to computethe LEM elasticity solution using a CMA formulation, is that∗ Institute of Applied Mathematics† Dept. of Computer Science, 2366 Main Mall, Vancouver, B.C., V6T1Z4, Canada, {djames|pai}@cs.ubc.cathe capacitance matrix 1 is the main quantity of interest: it is thecompliance matrix which relates the force feedback response to theimposed contact displacements. Also, the precomputation of GFseffectively decouples the global deformation and force responsecalculations, so that the capacitance matrix can be extracted fromthe the GFs at no extra cost; this is the fundamental mechanismby which a haptic interface can efficiently interact with a LEM ofvery large complexity. The user can feel no difference betweenthe force response of the complete system and the capacitance matrix,because none exists. Lastly, CMAs are direct matrix solverswhose deterministic operation count is appealing for real time applications.The second part of this paper addresses the special case ofpoint-like interaction. It has long been recognized that point contactis a convenient abstraction for haptic interactions, and thePHANToM TM haptic interface is a testament to that fact. Whileit is possible to consider the contact area to be truly a point for rigidmodels, infinite contact pressures are problematic for elastic modelsand tractions need to be distributed over finite surface areas. Wepropose to do this efficiently by introducing nodal traction distributionmasks which address at least two core issues. First, havinga point contact with force distributed over a finite area is somewhatcontradictory, and the traction distribution is effectively anunderdetermined quantity without any inherent spatial scale. Thisis resolved by treating the contact as a single displacement constraintwhose traction distribution enters as a user (or manipulandum)specified parameter. The distribution of force on the surfaceof the model can then be consistently specified in a fashion whichis independent of the scale of the mesh. Second, given the modelis discrete, special care must be taken to ensure a sufficiently regularforce response on the surface, since irregularities are very noticeableduring sliding contact motions. By suitably interpolatingnodal traction distributions, displacement constraints can be imposedwhich are consistent with regular contact forces for numerousdiscretizations.The remainder of the paper is organized as follows. After a discussionof related work (§2), the notation and definitions for a wideclass of linear elastostatic models used herein are given (§3). FastCMAs for general BVP solution using precomputed GFs of a particularreference BVP type are described in detail in §4. Particularattention is given to the role of the capacitance matrix for the constructionof globally consistent stiffness matrices for use in localhaptic buffer models (§5). The special case of point-like contactsare considered in detail, and we introduce (runtime computable)vertex masks for haptic presentation of surface stiffness (§6). Someresults are given for our implementations (§7) followed by conclusionsand a discussion of future work (§8).2 Related WorkWhile a significant amount of work has been done on interactivesimulation of physically-based elastic models, e.g., in computer1 The term “capacitance” is due to historical convention [16].1A141

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.orggraphics [14], only a relatively small subset of work has addressedthe simulation requirements of force feedback computer haptics.Much of the computer haptics literature on deformable models hasbeen concerned with surgical simulation, while our focus is moregeneral. It is not our intent to survey all related haptic work here,but simply to mention some relevant work combining kinestheticforce feedback with elastic models.2.1 Elastostatic ModelsThere are several instances in the literature of real time simulationof linear elastostatic models based on precomputed GFs methods.These models were used because of their low runtime costs, anddesirable force feedback properties. To date only polygonal modelsbased on FEM and BEM discretizations have been considered,although other variants are possible within the linear systems descriptionpresented in the following sections.Of particular relevance is the work done by researchers at INRIAwho have made extensive use of a real time elastostatic FEM modelfor liver related surgical simulations [6, 5, 9]. During a precomputationphase they have used condensation [35] as well as iterativemethods [8] to compute displacement responses due to unit forcesapplied to vertices on the “free” boundary. At run time, they solvea small system of equations to determine the correct superpositionof responses to satisfy the applied surface constraints, which maybe identified as a case of the capacitance matrix approach. We notethat the point-like contact approach used in [8] could benefit frompressure mask concepts (§6). Recently, they have used anisotropicmaterial properties for the liver [29]. Other groups have also usedthe precomputed elastostatic FEM approach of [6] for surgical simulation,e.g., the KISMET surgical simulator [23] incorporates precomputedmodels to provide high-fidelity haptic force feedback.A limitation of the GF precomputation strategy is that incrementalruntime modifications of the model require extra runtimecomputations. While it may be too costly for interactive applications,this can also be efficiently performed using low-rank updatingtechniques such as for static reanalysis in the engineering community[22]. For surgical simulation, a practical approach has beento use a hybrid domain decomposition approach in which a moreeasily modified dynamic model is used in a smaller region to becut [9, 17].Finally, the authors presented a interactive animation techniquein [21] which combined precomputed GFs of boundary elementmodels with matrix-updating techniques for fast boundary valueproblem (BVP) solution. Although computer haptics was an intendedapplication, no force feedback implementation was mentioned.This paper generalizes that approach with a broad GF-basedlinear systems framework that subsumes the discretization issues ofboth [21] and the FEM approaches of [6, 8].2.2 Other Elastic ModelsVarious approaches have been taken to simulate dynamic elasticmodels, by addressing commonly encountered difficulties such asthe computational complexity of time-stepping 3D models, and numericaltime integration issues, e.g., stiffness and stability. In orderto meet the intense demands of force feedback rendering rates,most have opted for a multirate simulation approach. It is worthnoting explicitly that methods for interactively simulating soft dynamicobjects are in many ways complementary to the CMA methodspresented here for simulating relatively stiff LEM. A few notableexamples are now mentioned.Local buffer models were presented by Balaniuk in [2] for renderingforces computed by e.g., deformable object, simulatorswhich can not deliver forces at fast rendering rates. An applicationof the technique was presented for a virtual echographic examtraining simulator in [10]. While we do not use the same approachhere, the local buffer model concept is related to our capacitancematrix method for force computation.Astley and Hayward [1] introduced an approximation for linearviscoelastic FEM models that also exploits linearity, in this case byprecomputing multilevel Norton equivalents for the system’s stiffnessmatrix. By doing so, haptic interaction is possible by employingan explicit multirate integration scheme wherein a modelassociated with the contact region is integrated at a higher rate thanthe remaining coarser model.Çavuşoǧlu and Tendick [7] also use a multirate approach. Balancedmodel reduction of a linearization of their nonlinear dynamicallumped element model is used to suggest a spatially localizeddynamic model approximation for force feedback rendering. Whilepromising, the example considered is a very special case of a supportedmodel, and it is unclear how the local model would be derivedin more generic geometric cases, as well as in the presenceof nonlocal influences such as for multiple changing contacts, e.g.,with surgical tools.Debunne et al. [11] presented a space-time approach for simulatinga hierarchical multirate dynamic linear-strain model. Zhuangand Canny [34] use a dynamic lumped finite element model exhibitingnonlinear (Green’s) strain. It is capable of being time-steppedat graphics frame rates for sufficiently soft objects using an explicitintegration scheme. Interactive simulation of dynamic elasticmodels exclusively for superquadric shapes was considered by Ramanathanand Metaxas [31]. Volumetric and voxel-based modelingapproaches for surgical simulation have also been considered, e.g.,by Gibson et. al. [13].3 Linear Elastostatic Boundary ModelPreliminariesLinear elastostatic objects are essentially three-dimensional linearsprings, and as such they are useful modeling primitives forphysically-based simulations. The unfamiliar reader might consulta suitable background reference before continuing [3, 18, 35, 4, 21].In this section, background material for a generic discrete GFdescription for a variety of precomputed linear elastostatic modelsis provided. Conceptually, GFs form a basis for describing allpossible deformations of a LEM subject to a certain class of constraints.This is useful because it (1) provides a common languageto describe all discrete LEMs, (2) subsumes extraneous discretizationdetails by relating only physical quantities, and (3) clarifies thegenerality of the force feedback algorithms described later.Another benefit of using GFs is that they provide an efficientmeans for exclusively simulating only boundary data (displacementsand forces) if desired. While it is possible to simulate variousinternal volumetric quantities (see §3.5), simulating only boundarydata involves less computation. This is sufficient since we are primarilyconcerned with interactive simulations that impose surfaceconstraints and provide feedback via surface deformation and contactforces.3.1 Geometry and Material PropertiesGiven that the fast solution method is based on linear systems principles,essentially any linear elastostatic model with physical geometricand material properties is admissible. We shall considermodels in three dimensions, although many arguments also applyto lower dimensions. Suitable models would of course includebounded volumetric objects with various internal material properties,as well as special subclasses such as thin plates and shells.Since only a boundary or interface description is utilized for specifyinguser interactions, other exotic geometries may also be easily2A142

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.orgconsidered such as semi-infinite domains, exterior elastic domains,or simply any set of parametrized surface patches with a linear response.Similarly, numerous representations of the surface and associateddisplacement shape functions are also possible, e.g., polyhedral,NURBS or subdivision surfaces [32].3.2 Nodal Displacements and TractionsLet the undeformed boundary be denoted by Γ. The change inshape of the surface is described by the surface displacement fieldu(x), x ∈ Γ, and the surface force distribution is called the traction2 field p(x), x ∈ Γ. We will assume that each surface field isparametrized by n nodal variables (see Figure 1), so that the discretedisplacement and traction vectors areu = [u 1, . . . , u n] T (1)p = [p 1 , . . . , p n ] T , (2)respectively, where each nodal value is a vector in R 3 . This descriptionadmits a very large class of surface displacement and tractiondistributions.uΓwhere x j ∈ Γ is the j th vertex. The traction field is a piecewiselinear function, and φ j(x) represents a “hat function” located at thej th vertex with φ j(x j) = 1. Given our implementation, we shalloften refer to node and vertex interchangeably.3.3 Discrete Boundary Value Problem (BVP)At each step of the simulation, a discrete BVP must be solvedwhich relates specified and unspecified nodal values, e.g., to determinedeformation and feedback forces. Without loss of generality,it shall be assumed that either position or traction constraints arespecified at each boundary node, although this can be extended toallow mixed conditions, e.g., normal displacement and tangentialtractions. Let nodes with prescribed displacement or traction constraintsbe specified by the mutually exclusive index sets Λ u andΛ p, respectively, so that Λ u ∩Λ p = ∅ and Λ u ∪Λ p = {1, 2, ..., n}.In order to guarantee an equilibrium constraint configuration wewill require that there is at least one displacement constraint, i.e.,Λ u ≠∅. We shall refer to the (Λ u, Λ p) pair as the BVP type.Typical boundary conditions for a force feedback loop consist ofspecifying some (compactly supported) displacement constraints inthe area of contact, with “free” boundary conditions (zero traction)and other (often zero displacement) support constraints outside thecontact zone. The solution to (7) yields the rendered contact forcesand surface deformation.Denote the unspecified and complementary specified nodal variablesbyv j ={ {pj : j ∈ Λ uūj : j ∈ Λand ¯vu j : j ∈ Λ j =u, (6)p ¯p j : j ∈ Λ pFigure 1: Illustration of discrete nodal displacements u defined atvertices on the undeformed boundary Γ (solid blue line), that resultin a deformation of the surface (to dashed red line). Althoughharder to illustrate, a similar definition exists for the traction vector,p.In order to relate traction distributions to forces, define a scalarfunction space, L, on the model’s boundary:L = span {φ j(x), j = 1 . . . n, x ∈ Γ} , (3)where φ j(x) is a scalar basis function associated with the j th node.The continuous traction field is then a 3-vector function with componentsin L,n∑p(x) = φ j(x)p j , (4)j=1The force on any surface area is equal to the integral of p(x) onthat area. It then follows that the nodal force associated with anynodal traction is given by∫f j = a jp j where a j = φ j(x)dΓx (5)defines the area associated with the j th node.Our implementation uses linear boundary element models, forwhich the nodes are vertices of a closed triangle mesh. The meshis modeled as a Loop subdivision surface [24] to conveniently obtainmultiresolution models for rendering as well as uniformly parameterizedsurfaces ideal for BEM discretization and deformationdepiction. The displacement and traction fields have convenientvertex-based descriptionsu j = u(x j), p j = p(x j),2 Surface traction describes force per unit area.Γrespectively. By linearity of the discrete elastic model, there formallyexists a linear relationship between all nodal boundary variables0 = Av + Ā¯v = Av − z (7)where the BVP system matrix A and its complementary matrix Āare, in general, dense block n-by-n matrices [18]. Body force termsassociated with other phenomena, e.g., gravity, have been omittedfor simplicity, but can be included since they only add an extra contributionto the z term.A key relationship between BVP system matrices (A, Ā) of differentBVP types (Λ u, Λ p) is that they are related by exchanges ofcorresponding block columns, e.g., (A :j , Ā :j ), and therefore smallchanges to the BVP type result in low-rank changes to the BVPsystem matrices (see §4.2.1).While the boundary-only system matrices in (7) could be constructedexplicitly, e.g., via condensation for FEM models [35] orusing a boundary integral formulation (see next section), it neednot be in practice. The discrete integral equation in Equation 7 isprimarily a common starting point for later definition of GFs andderivation of the CMA, while GFs may be generated with any convenientnumerical method, or even robotically scanned from realobjects [28].3.4 Example: Boundary Element ModelsA simple closed-form definition of (A, Ā) is possible for modelsdiscretized with the boundary element method (BEM) [4, 21];BEM discretizations are possible for models with homogeneousand isotropic material properties. The surface-based nodal quantitiesare related by the dense linear block matrix system0 = Hu − Gp =n∑n∑h iju j − g ij p j (8)j=1j=13A143

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.orgwhere G and H are n-by-n block matrices, with each matrix element,g ij or h ij, a 3-by-3 influence matrix with known expressions[4]. In this case, the j th block columns of A and Ā may beidentified as column exchanged variants of G and H:{−G:j : j ∈ ΛA :j =u(9)H :j : j ∈ Λ p{H:j : j ∈ ΛĀ :j =u(10)−G :j : j ∈ Λ pWhile we use BEM models for our implementation, we reiteratethat the CMA is independent of the method used to generate theGFs (explained next).3.5 Fast BVP Solution with Green’s FunctionsGFs of a single BVP type provide an economical means for solving(7) for that BVP, and when combined with the CMA (§4) will alsobe useful for solving other BVP types. From (7), the general solutionof a BVP type (Λ u, Λ p) may be expressed in terms of discreteGFs 3 asv = Ξ¯v =n∑ξ j¯v j = ∑j=1j∈Λ uξ j ū j + ∑j∈Λ pξ j¯p j , (11)where the discrete GFs of the BVP system are the block columnvectorsξ j = − ( A −1 Ā ) (12):jandΞ = −A −1 Ā = [ξ 1ξ 2 · · · ξ n] . (13)Equation (11) may be taken as the definition of the discrete GFs(and even (7)), since it is clear that the j th GF simply describes thelinear response of the system to the j th node’s specified boundaryvalue, ¯v j. Once the GFs have been computed for one BVP type,that class of BVPs may be solved easily using (11). An attractivefeature for interactive applications is that the entire solution can beobtained in 18ns flops 4 if only s boundary values (BV) are nonzero(or have changed since the last time step). Temporal coherence mayalso be exploited by considering the effect of individual changes incomponents of ¯v on the solution v.Further leveraging linear superposition, each GF system responsemay be augmented with other additional information in orderto simulate other precomputable quantities. Volumetric stress,strain and displacement data may also be simulated at preselectedlocations. Applications could use this to monitor stresses andstrains to determine, e.g., if fracture occurs or that a nonlinear correctionshould be computed.3.6 Precomputation of Green’s FunctionsSince the GFs for a single BVP type only depend on geometric andmaterial properties of the deformable object, they may be precomputedfor use in a simulation. This provides a dramatic speed-up forsimulation by determining the deformation basis (the GFs) aheadof time. While this is not necessary a huge amount of work (seeTable 2), the principal benefits for interactive simulations are theavailability of the GF elements via cheap look-up table operations,as well as the elimination of redundant runtime computation when3 Note on GF terminology: we are concerned with discrete numerical approximationsof continuous GFs, however for convenience these GF vectorswill simply be referred to as GFs.4 counting both + and ∗computing solutions, e.g., using a haptic device to grab a vertex ofthe model and move it around simply renders a single GF.Once a set of GFs for a LEM are precomputed, the overall stiffnesscan be varied at runtime by scaling BVP forces accordingly,however changes in compressibility and internal material distributionsdo require recomputation. In practice it is only necessary tocompute the GF corresponding to nodes which may have changingor nonzero boundary values during the simulation.4 Fast Global Deformation using CapacitanceMatrix Algorithms (CMAs)This section presents an algorithm for using the precomputed GFsof a relevant Reference BVP (RBVP) type to efficiently solve otherBVP types. With an improved notation and emphasis on computerhaptics, this section unifies and extends the approaches presentedin [21] exclusively for BEM models, and for FEM modelsin, e.g., [6], in a way that is applicable to all LEMs regardless ofdiscretization, or origin of GFs [28]. Haptic applications are consideredin §5.4.1 Reference Boundary Value Problem (RBVP)ChoiceA key step in the GF precomputation process is the initial identificationof a RBVP type, denoted by (Λ 0 u, Λ 0 p), that is representativeof the BVP types arising during simulations. For interactions withan exposed free boundary, a common choice is to have the uncontactedmodel attached to a rigid support as shown in Figure 2. Then-by-n block system matrices associated with the RBVP are identifiedwith a subscript as A 0 and Ā 0, and the corresponding GFs arehereafter always denoted by Ξ.Note that the user’s choice of RBVP type determines which typeof nodal constraints (displacement of traction) are commonly specified(in order to define Ξ), but is independent of the actual numericalboundary values ¯v used in practice. For example, there are norequirements that certain boundary values are zero, although thisresults in fewer summations (see (11)).Free Boundary;Fixed Boundary;0 Λ p0 Λ uFigure 2: Reference Boundary Value Problem (RBVP) example:The RBVP associated with a model attached to a flat rigid support isshown with boundary regions having fixed (Λ 0 u) or free (Λ 0 p) nodalconstraints indicated. A typical simulation would impose contactson the free boundary via displacement constraints with the CMA.4.2 Capacitance Matrix Algorithm (CMA) for BVPSolutionPrecomputed GFs speed-up the solution to the RBVP, but they canalso dramatically reduce the amount of work required to solve relatedBVP when used in conjunction with CMAs. This section describesthe CMA and presents the derivation of related formulae.4A144

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.org4.2.1 Relevant FormulaeSuppose the constraint-type changes, e.g., displacement↔traction,with respect to the RBVP at s nodes specified by the list of nodal indicesS = {S 1, S 2, . . . , S s}. As mentioned earlier, it follows from(6) and (7) that the new BVP system matrices (A, Ā) are related tothose of the RBVP (A 0, Ā 0) by s block column swaps. This maybe written asA = A 0 + ( Ā 0 − A 0)EETĀ = Ā 0 + ( A 0 − Ā 0)EETwhere E is an n-by-s block matrix]E =[I :S 1I :S 2· · · I :S s.(14)(15)containing columns of the n-by-n identity block matrix, I, specifiedby the list of updated nodal indices S. Postmultiplication byE extracts columns specified by S. Throughout, E is used to writesparse matrix operations using dense data, e.g., Ξ, and like the identitymatrix, it should be noted that there is no cost involved in multiplicationby E or its transpose.Since the BVP solution isv = A −1 z = −A −1 Ā¯v, (16)substituting (15) for Ā and the Sherman-Morrison-Woodbury formula[15] for A −1 (using the GF definition Ξ=−A −10 Ā0),A −1 = A −10 + (I + Ξ)E(−E T ΞE) −1 E T A −10 , (17)into (16), leads directly to an expression for the solution in terms ofthe precomputed GFs 5 . The resulting capacitance matrix formulaearev =}{{}v (0) + (E + (ΞE))} {{ }n × 1 n × s}{{}C −1s × sE} T {{v (0)}s × 1(18)where C is the s-by-s capacitance matrix, a negated submatrix ofΞ,C = −E T ΞE, (19)and v (0) is the response of the RBVP system to z=−Ā¯v,v (0) = A −10 z = [ Ξ ( I − EE T) − EE T] ¯v. (20)4.2.2 Algorithm for BVP SolutionWith Ξ precomputed, formulae (18)-(20) immediately suggest analgorithm given that only simple manipulations of Ξ and inversionof the smaller capacitance submatrix are required. An algorithm forcomputing all components of v is as follows:• For each new BVP type (with a different C matrix) encountered,construct and temporarily store C −1 (or LU factors) forsubsequent use.• Construct v (0) .• Extract E T v (0) and apply the capacitance matrix inverse to it,C −1 (E T v (0) ).• Add the s column vectors (E + (ΞE)) weighted byC −1 (E T v (0) ) to v (0) for the final solution v.5 Similarly from [21] with δA S =(Ā0−A 0 )E.4.2.3 Complexity IssuesGiven s nonzero boundary values, each new capacitance matrix LUfactorization involves at most 2 3 s3 flops, after which each subsequentsolve involves approximately 18ns flops (s ≪ n). This isparticularly attractive when s ≪ n is small, such as often occurs inpractice with localized surface contacts.An important feature of the CMA for interactive methods is thatit is a direct matrix solver with a deterministic operation count. It istherefore possible to predict the runtime cost associated with eachmatrix solve and associated force feedback subcomputations (see§5), thus making CMAs predictable for real-time computations.4.3 Selective Deformation ComputationA major benefit of the CMA direct BVP solver is that it is possibleto just evaluate selected components of the solution vector atruntime, with the total computing cost proportional to the numberof components desired. This is a key enabling feature for forcefeedback where, e.g., contact forces are desired at different ratesthan the geometric deformations. Selective evaluation would alsobe useful for optimizing (self) collision detection queries, avoidingsimulation of occluded or undesired portions of the model, as wellas rendering adaptive level of detail representations.In general, any subset of solution components may be determinedat a smaller cost than computing v entirely. Let the solutionbe desired at nodes specified by the set of indices D, with the desiredcomponents of v extracted by E T D. Using (18), the selectedsolution components may be evaluated asE T Dv = E T Dv (0) + E T D (E + (ΞE)) C −1 E T v (0)using only O(s 2 + s|D|) operations. The case where S = D isespecially important for force feedback and is discussed exclusivelyin the following section.5 Capacitance Matrices as Local BufferModelsFor force feedback enabled simulations in which user interactionsare modeled as displacement constraints applied to an otherwisefree boundary, the capacitance matrix has a very important role: itconstitutes an exact contact force response model by describing thecompliance of the contact zone. Borrowing terminology from [2],we say that the capacitance matrix can be used as a local buffermodel. While the capacitance matrix is used in §4.2.2 to determinethe linear combination of GFs required to solve a particularBVP and reconstruct the global deformation, it also has the desirableproperty that it effectively decouples the global deformationcalculation from that of the local force response. The most relevantbenefit for haptics is that the local contact force response may becomputed at a much faster rate than the global deformation.5.1 Capacitance Matrix Local Buffer ModelFrom (18), the S components of the solution v areE T v = E T [v (0) + (E + (ΞE)) C −1 E T v (0)]= E T v (0) + ( E T E )} {{ } C−1 E T v (0) + ( E T ΞE )} {{ } C−1 E T v (0)↓ I − C (from (19))= E T v (0) + C −1 E T v (0) − E T v (0)= C −1 ( E T v (0)) . (21)5A145

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.orgConsider the situation, which naturally arises in haptic interactions,in which the only nonzero constraints are updated displacementconstraints, i.e.,¯v = EE T¯v ⇒ v (0) = −¯v (using (20)). (22)In this case, the capacitance matrix completely characterizes thelocal contact response, since (using (22) in (21))E T v = −C −1 E T¯v. (23)This in turn parametrizes the global response since these components(not in S) are(I − EE T )v = (I − EE T )[v (0) + (E + (ΞE)) C −1 E T v (0)]= (I − EE T )(ΞE)(E T v) (24)where we have used (23) and the identity (I − EE T )E = 0. Suchproperties allow the capacitance matrix and Ξ to be used to deriveefficient local models for surface contact.For example, given the specified contact zone displacementsthe resulting tractions areu S = E T¯v, (25)p S = E T v = −C −1 ( E T¯v ) = −C −1 u S , (26)and the rendered contact force isf = a T S p S = ( −a T S C −1) u S = K S u S , (27)where K S is the effective stiffness of the contact zone used for forcefeedback rendering,a S = (a S1 , a S2 , . . . , a Ss ) T ⊗ I 3 (28)represents nodal areas (5), and I 3 is the scalar 3-by-3 identity matrix.A similar expression may be obtained for torque feedback.The visual deformation corresponding to solution components outsidethe contact zone is then given by (24) using p S =E T v.5.2 Example: Single Displacement ConstraintA simple case, which will be discussed in much greater detail in §6,is that of imposing a displacement constraint on single a node kwhich otherwise had a traction constraint in the RBVP 6 . The newBVP therefore has only a single constraint switch with respect tothe RBVP, and so s = 1 and S = {k}. The capacitance matrix hereis just C=−Ξ kk so that the k th nodal values are related byp k = −C −1 ū k = (Ξ kk ) −1 ū k or ū k = Ξ kk p k .The capacitance matrix can generate the force response, f = a k p k ,required for haptics in O(1) operations, and for graphical feedbackthe corresponding global solution is v =ξ k p k .5.3 Force Feedback for Multiple DisplacementConstraintsWhen multiple force feedback devices are interacting with themodel by imposing displacement constraints, the force and stiffnessfelt by each device are tightly coupled in equilibrium. Forexample, the stiffness felt by the thumb in Figure 3 will dependon how the other fingers are supporting the object. For multiplecontacts like this, the capacitance matrix again provides an efficientforce response model for haptics. Without presenting the equationsin detail, we shall just mention that the force responses for each ofthe contact patches can be derived from the capacitance matrix in amanner similar to equations (25)-(28).6 This case occurs, for instance, when the tip of a haptic device comesinto contact with the free surface of an object.Figure 3: Grasping simulation: Using a CyberTouch data input devicefrom Virtual Technologies Inc. (Top), a virtual hand (Bottom)was used to deform an elastostatic BEM model with approximately900 surface degrees of freedom (dof) at graphical frame rates (>30FPS) on a personal computer. The capacitance matrix algorithmwas used to impose displacement constraints on an otherwise freeboundary, often updating over 100 dof per frame. While force feedbackwas not present, the capacitance matrices computed could alsohave been used to render contact forces at a rate higher than that ofthe graphical simulation.6 Surface Stiffness Models for Point-likeContactThe second part of this paper concerns a special class of boundaryconditions describing point-like contact interactions. Such interactionsare commonly in the haptics literature for rigid surface models[26, 19]. Unlike their rigid counterparts, special care must betaken with elastic models to define finite contact areas for point-likeinteractions since point-like contacts defined only as single-vertex(§5.2) or nearest neighbour [8] constraints lead to mesh-related artifacts,and ambiguous interactions as the mesh is refined (see Figure4). However, the benefit of point-like contacts comes from theconvenience of the point-like parameterization of the contact andnot because the contact is highly concentrated or “pin-like”. Wepresent an approach using vertex pressure masks which maintainsthe single contact description yet distribute forces on a specified6A146

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.orgscale. This allows point contact stiffnesses to be consistently definedas the mesh scale is refined, and provides an efficient methodfor force feedback rendering of forces with regular spatial variationon irregular meshes.supported scalar function ρ(x) specified at each vertex on the freeboundary. The corresponding discrete vertex mask ρ may then bedefined using nodal collocation (see Figure 5),ρ j ={ρ(xj), j ∈ Λ 0 p,0, j ∈ Λ 0 u.,followed by suitable normalization,Figure 4: Point contact must not be taken literally for elastic models:This figure illustrates the development of a displacement singularityassociated with a concentrated surface force as the continuumlimit is approached. In the left image, an upward unit force appliedto a vertex of a discrete elastic model results in a finite vertexdisplacement. As the model’s mesh is refined (middle and right image),the same concentrated force load eventually tends to producea singular displacement at the contact location, and the stiffness ofany single vertex approaches zero (see Table 1). Such point-likeconstraints are mathematically ill-posed for linear models based ona small-strain assumption, and care must be taken to meaningfullydefine the interaction.6.1 Vertex Pressure Masks for Distributed PointlikeContactsIn this section, the distribution of force is described usingcompactly-supported per-vertex pressure masks defined on the freeboundary in the neighbourhood of each vertex.6.1.1 Vertex Pressure Mask DefinitionScalar pressure masks provide a flexible means for modeling vectorpressure distributions associated with each node. This allowsa force applied at the i th node to generate a traction distributionwhich is a linear combination of {φ j(x)} and not just φ i(x).In the continuous setting, a scalar surface density ρ(x) : Γ → Rwill relate the localized contact force f to the applied traction p via 7p(x) = ρ(x)fwhich in turn implies the normalization condition∫ρ(x)dΓx = 1. (29)ΓIn the discrete setting, the piecewise linear surface density on Γ isρ(x) =n∑φ j(x)ρ j ∈ L, (30)j=1ρ :=to ensure the satisfaction of (31).0ρ(x)ρa T ρ ,Figure 5: Collocated scalar masks: A direct means for obtaining arelative pressure amplitude distribution about each node, is to employa user-specified scalar functional of the desired spatial scale.The scalar pressure mask is then given by nodal collocation (left),after which the vector traction distribution associated with a nodalpoint load is then computed as the product of the applied force vectorand the (compactly supported) scalar mask (right).In the following, denote the density mask for the i th vertex bythe n-vector ρ i , with nonzero values being indicated by the set ofmasked nodal indices M i. Since the intention is to distribute forceon the free boundary, masks will only be defined for i ∈ Λ 0 p. Additionally,these masks will only involve nodes on the free boundary,M i ⊂Λ 0 p, as well as be nonempty, |M i| > 0.6.1.2 Example: Spherical Mask FunctionalsSpherically symmetric radially decreasing mask functionals witha scale parameter were suitable candidates for constructing vertexmasks via collocation on smooth surfaces. One functional we used(see Figure 6 and 7) had linear radial dependence,ρ i (x; r) =where r specifies the radial scale 8 .shown in Figure 6.{1 −|x−x i |r, |x − x i| < r,0, otherwise.pThe effect of changing r is,fand is parameterized by the discrete scalar vertex mask vector,ρ = [ρ 1, ρ 2, . . . , ρ n] T .Substituting (30) into (29), the discrete normalization condition satisfiedbecomesa T ρ = 1, (31)where a are the vertex areas from (5). Notice that the mask density1ρ has units of . areaIn practice, the vertex pressure mask ρ may be specified in a varietyof ways. It could be specified at runtime, e.g., as the byproductof a physical contact mechanics solution, or be a user specifiedquantity. We shall consider the case where there is a compactly7 Tensor-valued masks for torque feedback can also be computed.Figure 6: Illustration of changing mask scale: An exaggeratedpulling deformation illustrates different spatial scales in two underlyingtraction distributions. In each case, pressure masks weregenerated using the linear spherical mask functional (see §6.1.2) fordifferent values of the radius parameter, r.8 r may be thought of as the size of the haptic probe’s tip.7A147

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.org(a) a(x) (b) ‖K(x)‖ (c) masked ‖K(x)‖Figure 7: Effect of pressure masks on surface stiffness: Even models with reasonable mesh quality, such as this simple BEM kidney model,can exhibit perceptible surface stiffness irregularities when single-vertex stiffnesses are used. A plot (a) of the vertex area, a, clearly indicatesregions of large (dark red) and small (light blue) triangles. In (b) the norm of the single-vertex surface stiffness, ‖K(x)‖, reveals a noticeabledegree of mesh-related stiffness artifacts. On the other hand, the stiffness plotted in (c) was generated using a pressure mask (collocated linearsphere functional (see §6.1.2) of radius twice the mesh’s mean edge length) and better approximates the regular force response expected ofsuch a model. Masks essentially provide anti-aliasing for stiffnesses defined with discrete traction distributions, and help avoid “soft spots.”6.2 Vertex Stiffnesses using Pressure MasksHaving consistently characterized point-like force loads using vertexpressure masks, it is now possible to calculate the stiffness ofeach vertex. In the following sections, these vertex stiffnesses willthen be used to compute the stiffness at any point on model’s surfacefor haptic rendering of point-like contact.6.2.1 Elastic Vertex Stiffness, K EFor any single node on the free boundary, i ∈ Λ 0 p, a finite forcestiffness, K i ∈R 3×3 , may be associated with its displacement, i.e.,f = K iu i, i ∈ Λ 0 p.As a sign convention, it will be noted that for any single vertexdisplacementu i · f = u i · (K iu i) ≥ 0,i ∈ Λ 0 pso that positive work is done deforming the object.Given a force f applied at vertex i ∈ Λ 0 p, the corresponding distributedtraction constraints arewhere ζ i is the convolution of the GFs with the mask ρ, and characterizesthe distributed force load. The limiting case of a singlevertex constraint corresponds to M i ={i} with ρ i j =δ ij/a i so thatthe convolution simplifies to ζ i =ξ i/a i.Mesh Level Vertices ‖K‖ F , Single ‖K‖ F , Masked1 34 7.3 13.32 130 2.8 11.83 514 1.1 11.2Table 1: Vertex stiffness dependence on mesh resolution: This tableshows vertex stiffness (Frobenius) norms (in arbitrary units) atthe top center vertex of the BEM model in Figure 10(a), as geometricallymodeled using Loop subdivision meshes for three differentlevels of resolution. The stiffness corresponding to a single vertexconstraint exhibits a large dependence on mesh resolution, and hasa magnitude which rapidly decreases to zero as the mesh is refined.On the other hand, the stiffness generated using a vertex pressuremask (collocated linear sphere functional (see §6.1.2) with radiusequal to the coarsest (level 1) mesh’s mean edge length) has substantiallyless mesh dependence, and quickly approaches a nonzerovalue.p j = ρ i jf.Since the displacement of the i th vertex isu i = ∑j∈M iρ i jΞ ijf,therefore the effective elastic stiffness of the masked vertex is⎛ ⎞K i = K E i = ⎝ ∑ρ i jΞ ij⎠j∈M i−1, i ∈ Λ 0 p. (32)Some examples are provided in Table 1 and Figure 7.Therefore, in the simple case of a single masked vertex displacementconstraint u i, the local force response model exactly determinesthe resulting force, f = K iu i, distributed in the masked region.The corresponding globally consistent solution is⎛v = ζ if = ⎝ ∑ ⎠ fj∈M iρ i jξ j⎞6.2.2 Rigid Vertex Stiffness, K RFor rigid surfaces a finite force response may be defined using anisotropic stiffness matrix,K R = k Rigid I 3 ∈ R 3×3 , k Rigid > 0.This is useful for defining responses at position constrained verticesof a deformable model,K i = K R , i ∈ Λ 0 u, (33)for at least two reasons. First, while it may seem physically ambiguousto consider contacting a constrained node of a deformableobject, it does allow us to define a response for these vertices withoutintroducing other simulation dependencies, e.g., how the hapticinteraction with the elastic object support is modeled. Second, weshall see in §6.3 that defining stiffness responses at these nodes isimportant for determining contact responses on neighbouring triangleswhich are not rigid.8A148

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.org6.3 Surface Stiffness from Vertex StiffnessesGiven the vertex stiffnesses, {K i} n i=1, the stiffness of any locationon the surface is defined using nodal interpolationK(x) =n∑φ i(x)K i, x ∈ Γ, (34)i=1so that (K(x)) kl∈ L. Note that there are no more than threenonzero terms in the sum of (34), corresponding to the vertices ofthe face in contact. In this way, the surface stiffness may be continuouslydefined using only |Λ 0 p| free boundary vertex stiffnesses anda single rigid stiffness parameter, k Rigid , regardless of the extent ofthe masks. The global deformation is then visually rendered usingthe corresponding distributed traction constraints.For a point-like displacement constraint ū applied at x ∈ Γ on atriangle having vertex indices {i 1, i 2, i 3}, the corresponding globalsolution is∑v =ζ iφ i(x)f. (35)i∈{i 1 ,i 2 ,i 3 }∩Λ 0 pThis may be interpreted as the combined effect of barycentricallydistributed forces, φ i(x)f, applied at each of the triangle’s threemasked vertex nodes, which is consistent with (38).6.4 Rendering with Finite Stiffness Haptic DevicesSimilar to haptic rendering of rigid objects, elastic objects withstiffnesses greater than some maximum renderable magnitude (dueto hardware limitations) have forces displayed as softer materialsduring continuous contact. This can be achieved using a haptic vertexstiffness, K H i , which is proportional to the elastic vertex stiffness,K E i . While the stiffnesses could all be uniformly scaled on the freeboundary, this can result in very soft regions if the model has a widerange of surface stiffness. Another approach is to set( )K H i = η iK E i where η i = min 1, ‖KR ‖,‖K E i ‖so that the elastic haptic model is never more stiff than a rigid hapticmodel. The surface’s haptic stiffness K H (x) is then determinedusing (34), so that ‖K H (x)‖ ≤ ‖K R ‖, ∀x ∈ Γ.In accordance with force reflecting contact, the deformed elasticstate corresponds to the haptic force applied at the contact locationx C . This produces geometric contact configurations similarto that shown in Figure 8, where the haptic displacement u Hcan differ from the elastic displacement u E . The geometric deformationis determined from the applied force f and equation (35).Note that when the haptic and elastic stiffnesses are equal, suchas for soft materials, then so are the elastic and haptic displacements.In all cases, the generalized “god object” [36] or “surfacecontact point” [33] is defined as the parametric image of x C on thedeformed surface.7 Experimental ResultsGFs were precomputed using the boundary element method (BEM)with piecewise linear boundary elements. Table 2 provides timingsfor the BEM precomputation stages as well as the submillisecondcost of simulating point-like deformations using GFs. Further timingsof CMA suboperations are shown in Table 3, and reflect interactiveperformance for modest numbers of constraint type changes,s. All timings were performed using unoptimized Java code ona single processor Pentium III, 450MHz, 256MB computer withn Cx CFigure 8: Geometry of point-like contact: The surface of thestatic/undeformed geometry (curved dashed line) and that of the deformedelastic model (curved solid line) are shown along with: appliedforce (f), static contact location (x C ), deformed elastic modelcontact location (x E ), haptic probe-tip location (x H ), haptic contactdisplacement (u H = x H − x C ), elastic contact displacement(u E =x E −x C ), static contact normal (n C ) and elastic contact normal(n E ). Once the contact is initiated by the collision detector, the slidingfrictional contact can be tracked in surface coordinates at forcefeedback rates.Sun’s JDK 1.3 client JVM for Windows. These times can be significantlyreduced by using hardware-optimized matrix libraries, andcurrent computing hardware.An application of the CMA for multiple distributed contacts withunilateral contact constraints was the grasping task illustrated inFigure 3 using the LEM from Figure 10(a). A short video clip isalso available online [20].Our current force feedback implementation is based on the pointlikecontact approach discussed in the previous section. Forcesare rendered by a 3 dof PHANToM TM haptic interface (model 1.0Premium), on a dual Pentium II computer running Windows NT.The haptic simulation was implemented in C++, partly using theGHOST c○ toolkit, and interfaced to our ARTDEFO elastostatic objectsimulation written in Java TM and rendered with Java 3D TM . Thefrictional contact problem is computed by the haptic servo loop at1 kHz, which then prescribes boundary conditions for the slowergraphical simulation running at 25–80 Hz. For a point-like contact,it was only necessary to perform collision detection on theundeformed model, so this was done using the GHOST c○ API. Aphotograph of the authors demonstrating the simulation is shownin Figure 9, and a number of screen shots for various models areshown in Figure 10. A short video clip is also available online [20].We observed that the vertex masks were successful in producingnoticeable improvements in the smoothness of the sliding contactforce, especially when passing over regions with irregular triangulations(see Figure 7). We have not conducted a formal humanstudy of the effectiveness of our simulation approach. However,the haptic simulation has been demonstrated to hundreds of usersat two conferences: the 10 th Annual PRECARN-IRIS (Institute forRobotics and Intelligent Systems) Conference (Montreal, Quebec,Canada, May 2000) and in the ACM SIGGRAPH 2000 Exhibition(New Orleans, Louisiana, USA, July 2000). Users reported thatthe simulation felt realistic. In general, the precomputed LEM approachwas found to be both stable and robust.8 Summary and DiscussionWe have presented a detailed approach for real time solution ofboundary value problems for discrete linear elastostatic models(LEM), regardless of discretization, using precomputed GFs in con-fu En EExHuxH9A149

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.org(a) A simple nodular shape with a fixed base region.(b) A kidney-shaped model with position constrained vertices on part of the occluded side.(c) A plastic spatula with a position constrained handle.(d) A seemingly gel-filled banana bicycle seat with matching metal supports.Figure 10: Screenshots from real time haptic simulations: A wide range of ARTDEFO models are shown subjected to various displacementsusing the masked point-like contacts of §6. For each model, the middle of the three figures is uncontacted by the user’s interaction point (asmall green ball).10A150

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.orgElastic Model # Vertices (n) # Faces Precomp (min) LUD % Simulate (ms)Nodule 130v (89 free) 256f 1.1 1.1% 0.05Kidney 322v (217 free) 640f 7.7 3.1% 0.13Spatula 620v (559 free) 1248f 45 5.7% 0.34Banana Seat 546v (245 free) 1088f 25 20.0% 0.15Table 2: GF precomputation and simulation times for the BEM models depicted in Figure 10. All GFs corresponding to moveable freevertices (in Λ 0 p) were computed, and the precomputation time (Precomp) of the largest model is less than an hour. As is typical of BEMcomputations for models of modest size (n < 1000), the O(n 2 ) construction of the matrices (H and G in equation 8) is a significant portionof the computation, e.g., relative to the O(n 3 ) cost of performing the LU decomposition (LUD %) of the A matrix. The last column indicatesthat submillisecond graphics-loop computations (Simulate) are required to determine the point contact deformation response of each model’sfree boundary.# Updates, s LU Factor (ms) LU Solve (ms) (ΞE)(E T¯v) for n=100 (ms)10 0.54 0.03 0.3820 2.7 0.15 0.7440 19 0.58 1.7100 310 5.7 5.7Table 3: Timings of CMA suboperations such as LU decomposition (LU Factor) and back-substitution (LU Solve) of the capacitance matrix,as well as the weighted summation of s GFs (per 100 nodes) are shown for different sizes of updated nodal constraints, s.Figure 9: : Photograph of simulation in use: Users were able topush, slide and pull on the surface of the model using a point-likemanipulandum. Additionally, it was possible to change the surfacefriction coefficient, as well as the properties of the pressure mask,with noticeable consequences. The PHANToM TM (here model 1.0Premium) was used in all force feedback simulations, and is clearlyvisible in the foreground.junction with capacitance matrix algorithms (CMAs). The datadrivenCMA formulation highlights the special role of the capacitancematrix in computer haptics as a contact compliance usefulfor generating contact force and stiffness models, and provides aframework for extending the capabilities of these models.Additionally, the important special case of point-like contact wasaddressed with special attention given to the consistent definitionof contact forces for haptics. While this topic has been discussedbefore, we have introduced vertex masks to specify the distributionof contact forces in a way which leads to physically consistent forcefeedback models which avoid the numerical artifacts which lead tononsmooth rendering of contact forces on discrete models, as wellsas ill-defined contacts in the continuum limit.There are several issues to be addressed by future work on thesimulation of LEMs for computer haptics.One of the promises of linear GF models is that it should be possibleto precompute and haptically touch large-scale models even ifthey are too large to be graphically rendered. However, the CMApresented here is very efficient for small models (small n) and limitedconstraints (small s), but further optimizations and required forprecomputing, storing and simulating large-scale LEMs. ExtendingLEMs to accomodate geometric and material nonlinearities is alsoan area of study. Results addressing these problems will appearshortly in subsequent publications.A key challenge for interactively rendering elastic models is theplausible approximation of friction in the presence of multiple distributedelastic-rigid and elastic-elastic contacts. While large contactareas are a potential problem for LEM haptics, i.e., due tolarge update costs, the accompanying collision detection and frictionproblems appear to be at least as difficult. Incorporation ofLEMs into hybrid interactive dynamical simulations is also a relativelyunexplored area.Finally, the same issues (perceptible force regularity and spatialconsistency) which motivated our approach for a single point-likecontact, also arise for multiple point-like contacts and in generalwith multiple distributed contacts. The tight coupling of force stiffnessesbetween all contact zones, and therefore each (networked)force feedback device, can make this a difficult problem in practice.AJustification of Interpolated TractionDistributions for Point ContactThis section derives the nodal boundary conditions associated witha localized point contact at an arbitrary mesh location. The practicalconsequence is that the discrete traction distribution may beconveniently interpolated from suitable nearby nodal distributionsor masks.Given a continuous surface traction distribution, p(x), a correspondingdiscrete distribution Φ(x)p may be determined by a suitableprojection into L of each Cartesian component of p(x). Forexample, consider the projection of a scalar function on Γ definedas the minimizer of the scalar functional E : R 3n ↦→ R,∫E(p) =Γ[‖p(x) − Φ(x)p‖22 + ‖BΦ(x)p‖ 2 ]2 dΓx,where B : L ↦→ R is some linear operator that can be used, e.g., to11A151

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.orgpenalize nonsmooth functions, and Φ(x) : R 3n ↦→ R 3 is a nodalinterpolation matrix defined on the surface,Φ(x) = [φ 1(x)φ 2(x) · · · φ n(x)] ⊗ I 3, x ∈ Γ,where I 3 is the 3-by-3 identity matrix. The Euler-Lagrange equationsfor this minimization are∑ n(∫j=1 [φi(x)φj(x) + (Bφi(x)) (Bφj(x))] dΓx) pΓ j= ∫ φi(x)p(x)dΓx i = 1, 2, . . . , n,Γwhich, in an obvious notation, is written as the linear matrix problemAp = f (36)to be solved for the nodal traction values p. Note that A has unitsof area.The relevant traction distribution for point-like contact is a scaleindependentconcentrated point loadp(x) = p δ (x) = f δ δ(x − x δ )which models a force f δ delivered at x δ ∈Γ. The force n-vector inequation (36) has componentsf i = φ i(x δ )f δand the corresponding pressure distribution’s nodal values arep = A −1 f.For compactly supported basis functions, φ i(x), f has only a smallnumber of nonzero components for any given x. Hence φ i(x δ ) arethe interpolation weights describing the contribution of the nearbynodal pressure distributions, here specified by the columns of A −1 .As an example, consider the important case where L is a continuouspiecewise linear function space with φ i(x j) = δ ij. This wasthe space used in our implementation. In this case, at most onlythree components of f are nonzero, given by the indices {i 1, i 2, i 3}which correspond to vertices of the contacted triangle τ δ , i.e., forwhich x δ ∈ τ δ . The values φ i(x δ ) are the barycentric coordinatesof x δ in τ δ . The pressure distribution’s nodal values are thenp = A −1 f (37)=3∑ (A−1 ) f:i ikk==k=13∑k=1[ (Aφ ik (x δ −1)) f δ]:i k3∑φ ik (x δ )p (ik) , (38)k=1where p (ik) is the pressure distribution corresponding to the applicationof the load directly to vertex i k , and () :ik refers to blockcolumn i k of the matrix. Therefore the piecewise linear pressuredistribution for a point load applied at a barycentric location on atriangle is equal to the barycentric average of the pressure distributionsassociated with the point load applied at each of the triangle’svertices. This may be recognized as an elastic generalizationof force shading [27] for rigid models.Note that the j th column of A −1 is a vertex mask that describesthe nodal distribution of the load applied to the j th vertex. Modifyingthe penalty operator B results in masks with varying degrees ofsmoothness and spatial localization.References[1] Oliver Astley and Vincent Hayward. Multirate haptic simulationachieved by coupling finite element meshes throughnorton equivalents. In Proceedings of the IEEE InternationalConference on Robotics and Automation, 1998.[2] Remis Balaniuk. Building a haptic interface based on a buffermodel. In Proceedings of the IEEE International Conferenceon Robotics and Automation, 2000.[3] James R. Barber. Elasticity. Kluwer Press, Dordrecht, firstedition, 1992.[4] C. A. Brebbia, J. C. F. Telles, and L. C. Wrobel. Boundary ElementTechniques: Theory and Applications in Engineering.Springer-Verlag, New York, second edition, 1984.[5] Morten Bro-Nielsen. Surgery simulation using fast finite elements.Lecture Notes in Computer Science, 1131:529–, 1996.[6] Morten Bro-Nielsen and Stephane Cotin. Real-time volumetricdeformable models for surgery simulation using finiteelements and condensation. Computer Graphics Forum,15(3):57–66, August 1996.[7] Murak Cenk Çavuşoǧlu and Frank Tendick. Multirate simulationfor high fidelity haptic interaction with deformableobjects in virtual environments. In Proceedings of the IEEEInternational Conference on Robotics and Automation, SanFrancisco, USA, 2000.[8] S. Cotin, H. Delingette, and N. Ayache. Realtime elastic deformationsof soft tissues for surgery simulation. IEEE TransactionsOn Visualization and Computer Graphics, 5(1):62–73, 1999.[9] Stephane Cotin, Herve Delingette, and Nicholas Ayache. Efficientlinear elastic models of soft tissues for real-time surgerysimulation. Technical Report RR-3510, Inria, Institut Nationalde Recherche en Informatique et en Automatique, 1998.[10] Diego d’Aulignac, Remis Balaniuk, and Christian Laugier. Ahaptic interface for a virtual exam of a human thigh. In Proceedingsof the IEEE International Conference on Roboticsand Automation, San Francisco, USA, 2000.[11] Gilles Debunne, Mathieu Desbrun, Alan Barr, and Marie-Paule Cani. Interactive multiresolution animation of deformablemodels. In N. Magnenat-Thalmann and D. Thalmann,editors, Computer Animation and Simulation ’99,SpringerComputerScience, pages 133–144. Springer-VerlagWien New York, 1999. Proceedings of the EurographicsWorkshop in Milano, Italy, September 7–8, 1999.[12] Y. Ezawa and N. Okamoto. High-speed boundary elementcontact stress analysis using a super computer. In Proc. of the4 th International Conference on Boundary Element Technology,pages 405–416, 1989.[13] Sarah Gibson, Christina Fyock, Eric Grimson, Takeo Kanade,Rob Kikinis, Hugh Lauer, Neil McKenzie, Andrew Mor, ShinNakajima, Hide Ohkami, Randy Osborne, Joseph Samosky,and Akira Sawada. Volumetric object modeling for surgicalsimulation. Medical Image Analysis, 2(2):121–132, 1998.[14] Sarah F. Gibson and Brian Mirtich. A survey of deformablemodels in computer graphics. Technical Report TR-97-19,Mitsubishi Electric Research Laboratories, Cambridge, MA,November 1997.12A152

Vol. 2, Number 1, Haptics-e, September 27, 2001http://www.haptics-e.org[15] Gene H. Golub and Charles F. Van Loan. Matrix Computations.Johns Hopkins University Press, Baltimore and London,third edition, 1996.[16] William W. Hager. Updating the inverse of a matrix. SIAMReview, 31(2):221–239, June 1989.[17] K. V. Hansen and O. V. Larsen. Using region-of-interest basedfinite element modeling for brain-surgery simulation. LectureNotes in Computer Science, 1496:305–, 1998.[18] Friedel Hartmann. The mathematical foundation of structuralmechanics. Springer-Verlag, New York, 1985.[19] C.-H. Ho, C. Basdogan, and M. A. Srinivasan. Efficient pointbasedrendering techniques for haptic display of virtual objects.Presence, 8(5):477–491, 1999.[20] Doug L. James. http://www.cs.ubc.ca/˜djames/deformable.[21] Doug L. James and Dinesh K. Pai. ARTDEFO: Accurate RealTime Deformable Objects. Computer Graphics, 33(AnnualConference Series):65–72, 1999.[22] A. M. Abu Kassim and B. H. V. Topping. Static reanalysis:a review. Journal of Structural Engineering, 113:1029–1045,1987.[23] U. Kühnapfel, H.K. Çakmak, and H. Maaß. 3d modeling forendoscopic surgery. In Proceedings of IEEE Symposium onSimulation, pages 22–32, Delft University, Delft, NL, October1999.[24] Charles Loop. Smooth subdivision surfaces based on triangles.Master’s thesis, University of Utah, Department ofMathematics, 1987.[25] K. W. Man, M. H. Aliabadi, and D. P. Rooke. Analysis ofContact Friction using the Boundary Element Method. InM. H. Aliabadi and C. A. Brebbia, editors, ComputationalMethods in Contact Mechanics, chapter 1, pages 1–60. ComputationalMechanics Publications and Elsevier Applied Science,1993.[26] T. H. Massie and J. K. Salisbury. The phantom haptic interface:A device for probing virtual objects. In ASME WinterAnnual Meeting, Symposium on Haptic Interfaces for VirtualEnvironment and Teleoperator Systems, Chicago, IL, Nov.1994.[27] Hugh B. Morgenbesser and Mandayam A. Srinivasan. Forceshading for haptic shape perception. In Proceedings of theASME Dynamics Systems and Control Division, volume 58,1996.[28] Dinesh K. Pai, Kees van den Doel, Doug L. James, JochenLang, John E. Lloyd, Joshua L. Richmond, and Som H. Yau.Scanning Physical Interaction Behavior of 3D Objects. InComputer Graphics Proceedings (SIGGRAPH 2001). ACMSiggraph, 2001.[29] G. Picinbono, J. C. Lombardo, H. Delingette, and N. Ayache.Anisotropic elasticity and force extrapolation to improve realismof surgery simulation. In Proceedings of IEEE InternationalConference on Robotics and Automation, San Francisco,USA, 2000.[30] William H. Press, Brian P. Flannery, Saul A. Teukolsky, andWilliam T. Vetterling. Numerical Recipes: The Art of ScientificComputing, chapter Sherman-Morrison and Woodbury,pages 66–70. Cambridge University Press, Cambridge, 1987.[31] R. Ramanathan and D. Metaxas. Dynamic deformable modelsfor enhanced haptic rendering in virtual environments. InIEEE Virtual Reality Conference, 2000.[32] P. Schröder, D. Zorin, T. DeRose, D. R. Forsey, L. Kobbelt,M. Lounsbery, and J. Peters. Subdivision for modeling andanimation. SIGGRAPH 99 Course Notes, August 1999.[33] Sensable Technologies, Inc. GHOST SDK,http://www.sensable.com.[34] Yan Zhuang and John Canny. Haptic interaction with globaldeformations. In Proceedings of the IEEE International Conferenceon Robotics and Automation, 2000.[35] O. C. Zienkiewicz. The Finite Element Method. McGraw-Hill Book Company (UK) Limited, Maidenhead, Berkshire,England, 1977.[36] C. B. Zilles and J. K. Salisbury. A constraint-based god-objectmethod for haptic display. In ASME Haptic Interfaces for VirtualEnvironment and Teleoperator Systems, volume 1, pages149–150, Chicago, IL (US), 1994.13A153

Scanning Physical Interaction Behavior of 3D ObjectsDinesh K. Pai, Kees van den Doel, Doug L. James, Jochen Lang, John E. Lloyd, Joshua L. Richmond, Som H. YauDepartment of Computer Science, University of British Columbia, Vancouver, Canada{pai,kvdoel,djames,jlang,lloyd,jlrichmo,shyau}@cs.ubc.ca(a) Real toy tiger. By design,it is soft to touchand exhibits significantdeformation behavior.(b) Deformable model oftiger scanned by our system,with haptic interaction.(c) Real clay pot, withglazed regions. The potexhibits a variety of contacttextures and sounds.Figure 1: Examples of behavior models scanned by our system(d) Virtual interactionwith scanned model ofpot; includes contact textureand contact sounds.AbstractWe describe a system for constructing computer models ofseveral aspects of physical interaction behavior, by scanningthe response of real objects. The behaviors we can successfullyscan and model include deformation response, contacttextures for interaction with force-feedback, and contactsounds. The system we describe uses a highly automatedrobotic facility that can scan behavior models of whole objects.We provide a comprehensive view of the modelingprocess, including selection of model structure, measurement,estimation, and rendering at interactive rates. Theresults are demonstrated with two examples: a soft stuffedtoy which has significant deformation behavior, and a hardclay pot which has significant contact textures and sounds.The results described here make it possible to quickly constructphysical interaction models of objects for applicationsin games, animation, and e-commerce.Keywords: Behavioral Animation, Deformations, Haptics, Multimedia,Physically Based Modeling, Robotics, Sound Visualization1 IntroductionReal 3D objects exhibit rich interaction behaviors which includehow an object deforms on contact, how its surface feelsPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copiesare not made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. To copyotherwise, to republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.ACM SIGGRAPH 2001, 12-17 August 2001, Los Angeles, CA, USA© 2001 ACM 1-58113-374-X/01/08...$5.00when touched, and what kinds of sounds it makes when oneinteracts with it. These aspects of visual, haptic 1 ,andauditorybehavior provide important interaction cues and increasethe sense of presence in virtual environments suchas games. Despite recent progress in deformation modeling(e.g., [37, 7, 18]), haptic rendering (e.g., [32]), and auditorydisplays (e.g., [12]), these aspects are either entirely missingfrom models used for computer graphics and interaction, ormust be painstakingly added by highly skilled professionals.We believe that a major reason for this unfortunate situationis the difficulty of constructing these complex multimodalmodels by hand. In this paper we show how this problemcan be solved by scanning the behavior of real objects.Constructing behavior models requires not only acquiringmeasurements of real object behavior, but also designingmathematical models whose parameters can be effectivelyestimated from the measurements, and can be effectivelyused for realistic rendering. Each of these steps is importantfor successfully modeling real object behavior. We give detaileddescriptions of how this can be done for three aspectsof interaction behavior: (1) deformation models which canbe rendered both visually and haptically; (2) contact texturemodels for capturing surface friction and roughness for hapticdisplay and dynamics simulation; and (3) contact soundmodels for synthesizing interaction sounds, including soundsof continuous interaction.Figure 1 shows some behavior models acquired by oursystem. Of course, it is somewhat difficult to show real timebehavior on paper. The accompanying video demonstratesthe behavior models better.In this paper we also describe how the acquisition of thesemodels can be automated using a highly integrated roboticmeasurement facility, and how behavior models can be registeredrelative to a geometric model of an object. Eventhough our facility is an expensive prototype and uses sophisticatedrobotics for interaction and behavior measurement,we believe it can be practical and economical for modeling1 haptics refers to the sense of touch.87A154

ecause the facility can be shared by multiple users. Thetechniques we describe can be used to construct a modelfoundry, similar to a VLSI chip foundry but in reverse. Userscould send real objects to such a foundry and receive in returna behavior model of the object.It is helpful to compare the work presented here to 3Dgeometric modeling, from which we draw inspiration and instruction.3D geometric models can be constructed by handusing 3D modeling software and for some special applications,this hand modeling and editing is essential. However,3D scanning technology has dramatically changed the waysuch models are constructed. 3D geometry scanning makesit possible to capture fine details for unprecedented realism,but more importantly, empowers users with modest artistictalents with the ability to construct 3D models quickly andinexpensively. In a similar way, we expect that hand constructionof interaction behavior will continue to be useful forsome applications (e.g., those requiring fine creative controlto match the context and narrative). But for many applicationsin games, animation, and e-commerce, the ability toconstruct highly detailed behavior quickly and inexpensivelyusing the techniques we describe here could transform theway we construct models for 3D object interaction.1.1 Related WorkWe are not aware of any other system capable of scanningmodels of contact interaction behavior such as that describedin this paper. However, each component of our system hasmany connections to previous work that we discuss in therelevant sections below. Here we briefly discuss related workin the general area of building models of 3D objects by measuringthe real world.Almost all the work in the area of modeling real world objectsin computer graphics has focused on geometric modelingand reflectance modeling. In recent years there has beendramatic progress in scanning geometric models of 3D objects(e.g., [17, 8, 33, 22, 23]). In part because of the highaccuracy of laser scanning, most of these techniques assumethat the range data are given and focus on estimating goodmeshes from the data. Techniques have also been developedfor geometry reconstruction from a few photographs [11] oreven from a simple desktop system with a wand [2]. Acquiringthe reflectance properties of existing objects has alsobeen an active research area (e.g., [33, 9, 31, 15]). Our workhas parallel goals with this type of automated model acquisition,but differs in terms of types of models acquired.Acquiring interaction models is more difficult because of thenecessity of actually interacting and making contact withthe objects to be measured.Our work has some connections with image based techniques[4] and motion capture in that a recording can beconsidered a simple model of behavior, which can be editedand transformed (e.g., [26]). With few exceptions they havegenerally not been used for estimating parameters of physicalmodels. They also do not account for inputs to the motion,and therefore can not be directly used for simulating theeffects of new inputs.Measuring the real world requires a certain amount of infrastructureand several groups have developed measurementfacilities for this purpose. We mention the Cornell LightMeasurement facility [15], the Columbia/Utrecht facilityused for constructing a reflectance and texture database [9],the CMU Virtualized Reality TM laboratory [41], the BerkeleyLight Stage [10]. Field-deployable systems includethe Digital Michelangelo project [23] and the IBM Pietàproject [31].Our own facility is perhaps the most highly automatedand flexible system available today. It was designed to provideone-stop shopping for a large number of measurementsrather than for highly accurate measurement. It requiredsignificant developments in robotics to control and coordinatethe various measurement and actuation subsystems ofthe facility. We have previously described the robotics aspectsof teleprogramming and motion planning for the system,and measurement techniques for sound and deformation(e.g., [25, 24]). However, the present paper is the firstto describe the complete process of modeling interaction behaviors,from real object to rendered model.2 A Framework for Modeling BehaviorTo help understand how to construct useful models of realobjects, it is helpful to break the task down into four steps:selection of model structure, measurement, parameter estimation,and rendering. In the following sections we detailhow these steps are carried out for acquiring models of deformation,contact texture, and sound.Selection of model structureThis defines the fundamental class of mathematical modelsthat will be used to represent real physical behavior. Inthis step we fix the structure and not the parameters of themodel.We emphasize that this step is a creative act of modeldesign, in which the modeler balances the competing needsof the accuracy of the model’s predictions, rendering speed,ease of acquiring measurements, stability of parameter estimation,etc. It is tempting to think that the model is dictatedby Physics and can be “looked up” in a textbook, butthis is far from the truth. In designing the model structureit is essential to realize that all models have a finite domainof applicability.MeasurementIn this step we acquire the data from the real world to constructthe model. This step is critical since all subsequentresults depend on it. It is non-trivial for several reasons.First, we are interested in contact behavior which can notbe simply observed but must be “excited” by physically interactingwith the object. Thus we not only need to measure,say, the sound produced by an object, but we also need away to hit the object in a carefully controlled manner to producethe sound. Second,traditional measurement techniquesare designed for measuring material properties of small samplesof objects, but we would like to build models of entireobjects without destroying them or changing their essentialglobal properties. Thus it is necessary to be able to moveeither the object or the measuring instruments or both, toaccess different parts of the object. Third, we need to registerthe different measurements relative to each other. Wedo this by registering all measurements relative to a geometricmodel of the object. For instance, this allows us to usethe surface roughness of a particular point on the surfaceto drive the sound produced by scraping the surface at thatpoint. However this means that we must first acquire a geometricmodel of the object prior to scanning the behavior.Finally, to interpret the raw data the instruments need tobe calibrated. We do this using special calibration objects,88A155

ut auto-calibration techniques could also be used. To facilitatethis kind of measurement we have built an integratedrobotic measurement facility described in §3.Parameter estimationEstimation connects measurements to models. It is importantto realize that estimation problems are frequentlyill-posed and can lead to very sensitive inverse problems.Therefore it is necessary to pre-process the raw data, useregularization (which can be done in a natural way usingBayesian priors), and to use robust estimation techniques.As an example, direct fitting of sound models to sound waveformsby least-squares techniques produces very poor resultsin the presence of noise. We describe robust techniques forestimation which we found work well for the various estimationproblems described below.RenderingFinally, the estimated models must be rendered, to producedeformations, haptic forces, and sounds. This step, ofcourse, is the primary motivation for the previous steps andinfluences design decisions throughout. In particular, sincewe are interested in interaction behavior, it is important thatthe models are rendered at interactive rates. We describeimplemented algorithms suitable for interactive rendering ofdeformation, forces, and sounds. The rendering step is alsoimportant for validation of the scanned model, i.e., determiningwhether the estimated model approximates realitysufficiently well for a given purpose. For computer graphicsand interaction, it is difficult to design sensible quantitativemetrics of model validity, and one must largely rely on userperception. We show results comparing the behavior of realobjects to simulations of the scanned behavior models.3 Measurement System for Interaction BehaviorCarefully acquiring the measurements required for modelbuilding is often the most challenging part of the modelingprocess. For acquiring the kind of contact measurements weneed, we have developed the UBC Active Measurement facility(ACME), a highly automated robotic measurement facility.The robotics aspects of the facility have been previouslydescribed in detail [25]. For completeness, we briefly outlineit here. We note, however, that the techniques described belowdo not depend on the use of this particular facility whichwas designed to provide a large variety of measurements; asimpler measurement set up could be constructed to buildspecific kinds of models. As with 3D geometry scanning, weexpect that in the future it may be possible to build portableor desktop versions of such a measurement system.Fig. 2 shows the facility which consists of a variety of sensorsand actuators, all under computer control. Its majorsubsystems include: a 3 DOF Test Station (bottom of image)used for precise planar positioning of the test object;a Field Measurement System (shown at the top left) consistingof a Triclops trinocular stereo vision system, a highresolution RGB camera, and a microphone, all mounted ona 5 DOF positioning robot; and a Contact MeasurementSystem (CMS, shown on the right) consisting of a Puma260 robot equipped with a force/torque sensor, mounted ona linear stage. The CMS is the key subsystem for contactmeasurements as it is used to interact with the object toFigure 2: ACME Facility Overviewbe modeled. The entire facility can be controlled from anylocation on the Internet. We will describe the use of the sensorsand actuators for measurement in the relevant sectionsbelow.4 Geometric ModelingThe geometric models required for registering other measurementscan be produced by any geometry scanning andmesh generation method (e.g., [17, 8, 33, 28, 23]). Whilethe focus of this paper is not on geometric modeling, forcompleteness we will briefly outline our approach to geometricmodel construction starting from stereo range measurements.Stereo range measurementThe main shape sensor in our measurement facility is atrinocular stereo vision system 2 which is capable of producinglarge amounts of viewpoint-registered range images atmodest resolutions (approx. 2-3 mm for a typical viewpoint),in close to real time. Accurate ranging relies on image featuresfor matching between the stereo cameras; additionalfeatures can be attached or projected onto the surface.Stereo range data is notoriously noisy, and for best resultsit is filtered in several ways. First, range is calculatedwith variable mask sizes in a voting scheme. We also utilizethe checks and filters of the Triclops stereo library. Furtherprocessing of the data by volumetric reconstruction requiresapproximate surface normals at range data points. The normalsare estimated by plane fitting in local image neighborhoods.Further filtering of the range-data is performedbased on the quality of fit, the number of valid range-dataper neighborhood and the viewing angle of the plane.Multresolution mesh constructionAn initial triangle mesh is generated from the filtered rangedata using a volumetric approach by reconstruction softwareprovided courtesy of NRC of Canada [28]. The number andquality of triangles depends on the surface sampling density.Further processing is required to arrive at a useful geometricmodel since this mesh may not be watertight; it may includesome of the supporting Test Station surface; and there may2 A Color Triclops from Point Grey Research.89A156

Figure 3: Clay Pot Mesh Reconstruction: (left) raw scanmesh (13150 vertices, 26290 faces) with various externalstray polygons; (middle) watertight base level (level 0) mesh(127 vertices, 250 faces) generated via simplification of theraw scan mesh; (right) level 3 subdivision connectivity mesh(8002 vertices, 16000 faces) generated using displaced subdivisionsurface style piercing of the raw scan mesh.be erroneous surfaces due to noisy data. An example isshown in the left image of Figure 3.Finally, we construct watertight subdivision connectivitytriangle meshes at several resolutions l =0, 1, 2,...,L becausethey are desirable for later modeling. Here we exploit acommon property of raw meshes produced from range data:while the region exterior to the object geometry may containundesirable mesh features, the interior of the mesh is afair representation of the scanned surface. This allows theconstruction of multiresolution meshes by expanding and refininga surface inside the object using a normal piercingprocess similar to the construction of displaced subdivisionsurfaces [21] and normal meshes [16]. The coarse resolutionmesh is generated by simplifying the raw mesh (usingQSlim [13]) and possibly some minor editing to ensure thatthe mesh is free of defects such as in regions with poor rangedata. To ensure that the mesh expansion process is bounded,the raw mesh is manually closed if the NRC package producesa large hole on the unimaged underside of the object;this was addressed for the tiger mesh by simply inserting alarge horizontal triangle.While this approach to mesh construction is not robust,it produces meshes of sufficient quality that we were able toproceed with our main physical modeling objectives.Texture mappingTexture mapping is accomplished using calibrated color imagesfrom the stereo vision system. The vertices of themeshes are projected into the color images using the pinholecamera calibration matrix, and triangles are tagged as visibleif their vertices and centroid are all visible. We select theimaged triangular area as the texture map if the product ofthe image area times the cosine between view direction andtriangle normal is maximum in all our images. We also adjustthe relative global brightness of all color texture images.The texture maps could be readily improved by blending ofthe local textures over the entire map eliminating abruptbrightness changes at edges, as in [27].5 Deformation ModelingSelection of model structureWe use linear elastostatic models for deformable objectssince these models can be simulated at interactive rates [18],and linearity makes parameter estimation easier. Most important,linear elastic models can be characterized usingGreen’s functions, which capture the input-output behaviorof the object’s boundary for a given set of boundaryconditions; therefore there is no need to observe or estimatequantities in the interior of the object.It is a practical consideration that the model must be fixturedfor it to be deformed by ACME’s robots. Therefore westart by assuming that the physical object will be measured(and rendered) while attached to a support, like the TestStation. The linear elastostatic model then approximatesthe displacement response u = u (l) of the resolution l meshvertices due to applied surface tractions 3 p=p (l) byu = Up or u (l) = U (l) p (l) . (1)Here u and p are block vectors of length n with 3-vector elements,where the displacement and traction at the k th vertexis u k and p k , respectively; U is a square n (l) -by-n (l) block matrixwith 3-by-3 matrix elements. The k th block column U :kdescribes the displacement response contribution from thek th applied traction p k . The diagonal blocks U kk describethe self-compliance of the k th vertex, and play an importantrole in defining vertex stiffnesses for force-feedback rendering[19]. The traction vector at a vertex is the force overthe area of the one-ring of a vertex. The force is distributedlinearly over the area as a hat function located at the vertex.The key behavior of the model is characterized by thedisplacement response of the unfixtured free surface due toforces applied to the free surface. This corresponds to theonly interesting measurable portion of the U matrix, whichin turn is related to the discrete Green’s function matrix [19]for the free boundary. For instance, rows of U correspondingto vertices attached to the fixtured support necessarily havezero displacement values; these same vertices correspond tocolumns which are not exposed and therefore can not beactively inspected.Finally, once the Green’s function matrix for a fixture configurationis known, deformations of the object can be simulatedefficiently using fast matrix updating techniques [19].MeasurementObjects to be measured, such as the stuffed toy tiger shownin Figure 1, are fixed to the Test Station. Deformation measurementsrecord surface displacements and contact forcesresulting from active probing with ACME’s CMS robot arm.During deformation the position of the contact probe can becontinuously monitored at 100 Hz. The robot arm’s wristforce sensor is capable of recording the six-dimensional forcetorquewrench at the tip of the probe at 1000 Hz.The robot probes surface locations corresponding to verticesof the geometric model at the desired reconstructionresolution 4 l. The position of the contact probe measuresthe displacement of the contacted vertex u k , while the forcetorquesensor measures the contact force. The effective contacttraction p k is the force divided by the effective vertexarea (one third of the sum of adjacent triangle areas). Sincethere are no other applied tractions on the free or fixed surfaces,the complete traction vector p is zero except for thesingle contact traction p k . In the following we describe howthe deformation of the free surface is measured and mappedonto the vertices in order to estimate u.3 Traction is the force per unit area, similar to pressure.4 To avoid measurement artifacts the scale of the mesh is largerthan the measurement probe’s contact area.90A157

the camera close to the object (≈ 0.8m). The measured displacementu covers surface area visible from a chosen viewpoint. In the estimation section below, we discuss how tocombinemultiplemeasurementsforthesamecontactvertex.Figure 4: 2D Flow on tiger, poked near the neck.The surface deformation of the object is measured visuallyusing the Triclops trinocular stereo-vision system in theField Measurement System (FMS) of ACME. The measurementis based on tracking visual surface features in threespatial dimensions in a range-flow [42] approach. Severalmethods for calculating dense range-flow from stereo andoptical flow exist [41, 43, 34].Our method utilizes redundancy in the imagery from thetrinocular stereo-system to increase robustness [20]; It issummarized as:• Segment image into “object surface” and “other,” basedon the geometric model of the undeformed object.• Calculate range from stereo.• Calculate simultaneous optical flow in images from eachcamera for the “object surface”.• Combine optical flow by voting and map the result intothree dimensions based on range data.• Use the “range-flowed” object surface to segment thenext image in the sequence and continue with stereocalculations as above.The optical flow during deformation measurement is shownin Figure 4. In our approach, the surfaces need sufficientvisual texture because of the reliance on stereo vision andoptical flow. Most objects have sufficient visual texture forstereo matching but if not non-permanent visual texture maybe applied (e.g., using pins, stickers, or water soluble paint).The next step in surface deformation measurement is themapping of range flow vectors to displacements of verticesof a chosen mesh level. Flow vectors are associated with astart position on a triangular patch of the undeformed mesh.We estimate the displacement of a vertex with a robust averagingprocess rather than just using the closest flow vectorto a vertex (see, for example, [1] for a discussion on robustmotion estimation). The flow vectors associated with triangularpatches joined at a vertex are median filtered. Thisis followed by a weighted average based on distance fromthe vertex. The flow vectors have to be dense enough onthe surface relative to the mesh level for this process to berobust. In our set-up, we get high density by positioningParameter estimationOur approach to the estimation of the matrix U from themeasurements of displacement u and traction p addressestwo main issues: (1) noise in the form of outliers in the displacementmeasurement and (2) incomplete measurements.Outliers can, on occasion, still be observed in the displacementmeasurement despite the above described filtering process.These outliers originate from consistent incorrect flowvectors due to mismatches in the range-flow or in the stereoprocessing over an image area. Incomplete measurementsarise from partially observed object surfaces, from reachabilitylimits of the CMS and from anisotropic responses ofthe object to probing.A single block element U ij describes the relationship betweenthe displacement u i and a single applied traction p j :u i = U ijp j .For each element, we obtain m≤M measurements by probingeach vertex location M times. We arrive at a standardleast squares problem to be solved for U T ij:[ ] p1Tj p2 j ···pm j UTij = [ ]u 1 i u 2 i ···u m TiWe solve this least squares problem if our measured tractionsare a response to a set of probe directions which span3-space. This guarantees that we excite the model in all directions.However, because of the possibility of (in general)noncompliant responses combined with the limited resolutionof the measurements, this does not guarantee that thesolution (U ij) T has full rank. Therefore, we calculate thesolution by means of the truncated singular value decomposition(TSVD). We truncate the singular values at theapproximate resolution of the traction measurements. Furthermore,we select a subset of measurements if we observean unsatisfactory fit of the estimated (U ij) T and repeat.This subset selection process is performed using the leasttrimmed squares (LTS) method [29], implemented efficientlyas in [30]. This process is robust even if (M − 4)/2 measurementsare spoiled.Figure 5: Plots of estimated displacement responses: (left)Missing observations result in unestimated response components(shown in black); the remaining nodes are color codedwith red indicating the greatest displacement and blue theleast. (right) These values are estimated by an interpolatingreconstruction to obtain the final deformation responses.At this stage of the estimation process, the measureddisplacement field columns of U still contain unestimated91A158

elements due to missing observations of the deformed surface(shown in Figure 5). This problem can be minimizedby obtaining more measurements but not entirely avoided.Scattered data reconstruction is used to fill in elements foreach column individually. We currently interpolate missingdisplacements by solving Laplace’s equation over the set ofunestimated vertices, but better methods are currently beinginvestigated. The result of this interpolation process isshowninFigure5.Finally, in order to improve rendering quality and reducemeasurement and estimation time we exploit the multiresolutionmesh structure to optionally infer Green’s functionresponses for unmeasured vertices. This is done by activelypoking the model at a resolution (l − 1) one level coarserthan the resolution l used to estimate displacement fields(illustrated in Figure 6). The k th odd vertex on level l hasaresponseU :k inferred if both even vertices (k 1,k 2)ofitsparent edge have responses. If so, the k th response U :k is linearlyinterpolated from the two parent responses, (U :k1 , U :k2 ).The local responses, U kk and U jk when vertex j is a one-ringneighbor of k, are handled differently.Unlike long range displacement influences which aresmoothly varying, these local values are associated with acusp in the displacement field. Simple interpolation for thesevalues is biased and leads to incorrect contact forces duringrendering. Instead, the local values are computed as theweighted average of parent responses which have had theirlocal parameterizations smoothly translated from even vertexk ∗ to odd vertex k, e.g., U kk is linearly interpolated from(U k1 k 1,U k2 k 2)not(U kk1 ,U kk2 ). This shifting of the parent’slocal response before averaging yields a good estimator ofthe local response at vertex k. The resulting displacementfield U :k is also linearly independent of U :k1 and U :k2 .By design, the Green’s function models can be rendered atinteractive rates using the algorithm described in [18, 19].Contact forces are rendered using a PHANToM forcefeedbackinterface with contact force responses computedusing vertex pressure masks [19]. The multiresolution deformingsurface is also displacement mapped using displacedsubdivision surfaces [21] to add extra geometric detail to themodel. Figure 1 and the accompanying video (and the CALdemonstration) show interaction with scanned tiger modelusing the PHANToM. In general the results are quite satisfactory,capturing non-local effects such as the movementof the head when the back of the tiger is poked. The modeldoes show some of the limitations of the linear model structurefor large input displacements, with somewhat exaggerateddeformations. For moderate input displacements 5 ,thescanned model behaves quite realistically.6 Contact TextureSelection of model structureContact texture denotes the way an object feels when it isrubbed or scraped. The two principal aspects we focus onin this paper are friction and surface roughness.For modeling friction, we use the standard “textbook”Coulomb friction modelf f = −µ ‖f n‖u m (2)where f f is the frictional force, µ is the coefficient of friction(and the model parameter to be determined), f n is thenormal force, and u m is a unit vector in the direction ofmotion.Surface roughness is a more elusive property and wholebooks have been written on how to model it [38]. Roughnessis usually associated with small-scale variations in thesurface geometry, which create variations in the tangentialcontact force proportional to the normal force. These tangentialforce variations can be modeled as local variations ˜µin the coefficient of friction. Combined with µ, this yieldsan effective coefficient of friction µ e for a given displacementx along some particular surface direction:µ e(x) =µ +˜µ(x).Figure 6: Multiresolution Mesh and Contact Sampling Pattern:(left) Coarse l = 0 parameterization of model, used foractive contact measurement, displayed on finest l = 2 displacedsubdivision surface mesh; (right) yellow points drawnon the l = 1 resolution mark the nodes at which the system’sdisplacement response to applied tractions was either measured(even vertices) or inferred (odd vertices).RenderingThis formulation has the advantage of unifying haptic renderingof friction and roughness, particulary with commercialhaptic devices like the PHANToM which implementtheir own contact and friction algorithms which may notcorrespond to the textbook model of Coulomb friction.Forces due to roughness tend to have some randomnessbut often also contain periodic components, particularly inhuman artifacts. We assume that the roughness is isotropicand that people are sensitive only to statistical features ofthe roughness force variation, and can not discern the specificwaveform. To capture both randomness and periodicitywe model the friction variations ˜µ(x) as an autoregressiveAR(p) process, driven by noise, so that˜µ(x) =˜µ(k∆) ≡ ˜µ(k) =p∑a i ˜µ(k − i)+σɛ(k)where k is the sample index, ∆ is the spatial discretization,σ is the standard deviation of the input noise, and ɛ(k) isazero-mean noise input with standard deviation of one. Themodel parameters to be determined are the a i and σ.The AR model is very suitable for real-time simulationand rendering, and typically one needs only a few parametersto reflect the roughness properties (as illustrated by Fig. 7).An AR(2) model is often sufficient in practice because itallows the modeling of a random sequence combined withone principal frequency.i=15 approximately < 15% of the tiger’s diameter92A159

0.250.20.150.10.0500 2 4 6 8 100.250.20.150.10.0500 2 4 6 8 10Figure 7: (left) Measured values of µ e for a 10 mm displacementalong a rough section of the clay pot shown in Fig. 8. (right)Simulation of µ e with µ =0.142 and ˜µ reproduced by an AR(2)model with a 1 = .783, a 2 = .116, and σ =0.0148.Measurement and EstimationFriction and surface roughness are noticeable in terms ofthe forces they produce on a contacting instrument. Hencewe can measure them in the same way: the robotic systemperforms a series of local rubs over the object surface witha probe (attached to a 6 DOF force sensor; Fig. 8) and theresulting force profiles are then analyzed.The object’s surface mesh representation is used to plan“where and how” to do the rubbing. At present, the systemassigns a contact texture model to each mesh vertex. This isdetermined either by explicit measurement, or by the interpolationof models at nearby vertices. The system employsa process of “active exploration”, in which models are initiallysampled over the object surface at a low resolution,with further, higher resolution sampling in areas where themodel parameters change significantly.Figure 8: (left) Robotic system rubbing the pot to determinecontact texture. (right) Surface friction map for the claypot, in which the brightness is scaled according to the localvalue of µ on the surface (with white corresponding to themaximum value of µ =0.5). The enameled portions of thepot, with µ ≈ 0.09, are clearly visible. The ornament on topof the pot was not sampled.The nominal friction coefficient µ is estimated first. If thesurface normal n were known accurately, one could directlydetermine f n and f f in Eq.(2) and use this to solve for µ.However, n is known only approximately (due to uncertaintyin the surface mesh and the actual contact location), andalso varies along the path. To compensate for this, we strokethe surface twice: once in a forward direction and once ina reverse direction. At any point along the path, we thenhave a force value f + which was measured during the forwardmotion, and another value f − which was measured duringthe reverse motion.f + and f − each have components parallel to the surfacenormal n, along with friction components f + f and f − f whichlie opposite to the motion directions and are perpendicularto n. Now even if n is unknown, and the magnitudes off + and f − differ, µ can still be estimated from the angle θbetween f + and f − :µ =tan(θ/2).This calculation is quite robust, as it is independent of travelspeed, the probe contact force and orientation, and of coursethe surface normal itself. By averaging the values of µ obtainedat various points along the path, a reasonable estimatefor µ over the whole path may be obtained. Our abilityto determine µ reliably is illustrated in Fig. 8.The values of n at each path point can also be estimatedfrom the direction of (f + f +f − f) and used to produce a smooth(typically quadratic) model of n(x) along the path.To calculate the effective friction, we use n(x) torelateµ eto the observed force values f acting on the probe tip:f = f n[n(x) − µ eu m(x)],where u m(x) is the direction of motion along the path and f nis the magnitude of the normal force. The unknowns in thisequation are f n and µ e.Solvingforµ e at every path point xyields µ e(x). An AR model is then fitted to ˜µ(x) =µ e(x)−µ,using autoregressive parameter estimation via the covariancemethod (e.g., the arcov function in MATLAB).RenderingTo demonstrate the rendering of contact texture, we useda PHANToM haptic device to implement a virtual environmentin which a user can rub an object with a point contact(represented as a red ball in the video). The GHOST softwaresupplied with the PHANToM was used to perform collisiondetection and to generate the corresponding contactand frictional forces.The friction value at the point of contact is generated byweighting the AR parameters at each vertex by the barycentriccoodinates in the triangle. The distance traveled alongthe surface divided by the spatial discretization ∆ of themeasurements determines the number of values to generateusing the AR model. The last 2 values generated are theninterpolated to obtain the effective coefficient of friction µ e.This value is passed to both the static and dynamic frictionparameters in GHOST.The resulting contact textures are quite convincing; it iseasy to distinguish between the different surface preparationsof the clay pot using the haptics alone. These resultsare best evaluated using the PHANToM haptic device (e.g.,in our CAL demo) though Figs. 7 and 8 give a good indicationof the sensitivity of our measurement technique.7 Sound ModelingSelection of model structureWe model the contact sounds of an object by filtering an excitation(the “audio-force”) through a modal resonator bankwhich models the sonic response of the object. The detailsof this technique are explained in [39]. For this we need toacquire both a modal resonance model of the object, whichwill depend on its shape and internal composition, and anexcitation model, which will depend mainly on the surfacestructure.93A160

The modal model M = {, , }, consists of a vector of length N whose components are the modal frequenciesin Hertz, a vector of length N whose components are the(angular) decay rates in Hertz, and an N × K matrix ,whose elements a nk are the gains for each mode at differentlocations. The modeled response for an impulse at locationk is given byN∑y k (t) = a nk e −dnt sin(2πf nt), (3)n=1for t ≥ 0(andiszerofort

dB200−20−40Frequency identificationmeasurement 1measurement 2averagepeaknoiseWe believe that the techniques described in this papercould greatly improve the way virtual environments and animationsare created. In addition to geometry and appearance,our methods will allow behaviors to be essential andeasily obtained properties of virtual objects. Our methodsmake it feasible to build compelling interactive virtual environmentspopulated with a large number of virtual objectswith interesting behavior.−60−80−1002 4 6 8 10 12 14 16 18 20 22KHzReferences[1] M.J. Black and P. Anandan. The Robust Estimationof Multiple Motions: Parametric and Piecewise-smoothFlow Fields. Computer Vision and Image Understanding,63(1):75–104, 1996.Figure 10: The power spectrum of two recorded impulse responses,their average, and the power spectrum of the backgroundnoise. The 20 most important peaks are indicated onthe graph. The “best” peaks are those considered to standout from their surrounding most clearly by “local height”when an object is excited at different locations. Suddenjumps in the timbre during scraping can be heard clearlyif we switch gain vectors discretely. We therefore utilize aform of “audio anti-aliasing”: at locations between meshvertices we smoothly interpolate the gains from the vertexgains, using the barycentric coordinates of the location inthe triangle. Note that because the gains a nk at differentvertices share the same modal frequencies, there is no needfor frequency interpolation.If the AR(2) filters turn out to be resons [35], we canscale the resonance frequency measured at a reference contactspeed with the actual contact speed in the simulation.This produces the effect of a shift in “pitch” dependent onthe sliding velocity. If the AR(2) filters are not resons (i.e.,if their poles are real), which will occur if there is no prominentcharacteristic length scale in the surface profile, thismodel does not produce the illusion of scraping at a changingspeed. The perceptual cue for the contact speed seems tobe contained in the shifting frequency peak. We have foundthat higher order AR models in such cases will find thesepeaks, but we have not completed this investigation at thetime of writing.If an audio-force wave-table is used, it is pitch-shifted usinglinear sample interpolation to correspond to the actualsimulation contact speed and the volume is adjusted proportionalto the power-loss as determined by friction andspeed [39].8 ConclusionsWe have described a system for modeling the interaction behaviorof 3D objects by scanning the behavior of real objects.Modeling interaction behavior is essential for creating interactivevirtual environments, but constructing such modelshas been difficult. We show how a variety of important interactionbehaviors, including deformation, surface texturefor contact, and contact sounds can be effectively scanned.We provided a description of the complete modeling processwhich could be used construct these types of models.We also described our own measurement facility which automatesmany of the steps in measuring contact interactionbehavior using robotics.[2] Y. Bouguet and P. Perona. 3D Photography on YourDesk. In Proc. ICCV98, pages 43–50, 1998.[3] J.C. Brown and M.S. Puckette. A High ResolutionFundamental Frequency Determination Based on PhaseChanges of the Fourier Transform. J. Acoust. Soc. Am.,94(2):662–667, 1993.[4] M. Cohen, M. Levoy, J. Malik, L. McMillan, andE. Chen. Image-based Rendering: Really New or DejaVu? ACM SIGGRAPH 97 Panel, pages 468 - 470.[5] P.R. Cook. Integration of Physical Modeling for Synthesisand Animation. In Proceedings of the InternationalComputer Music Conference, pages 525–528,Banff, 1995.[6] P.R. Cook and D. Trueman. NBody: Interactive MultidirectionalMusical Instrument Body Radiation Simulations,and a Database of Measured Impulse Responses.In Proceedings of the International Computer MusicConference, San Francisco, 1998.[7] S. Cotin, H. Delingette, J-M Clement, V. Tassetti,J. Marescaux, and N. Ayache. Geometric and PhysicalRepresentations for a Simulator of Hepatic Surgery.In Proceedings of Medicine Meets Virtual Reality IV,pages 139–151. IOS Press, 1996.[8] B. Curless and M. Levoy. A Volumetric Method forBuilding Complex Models from Range Images. In SIG-GRAPH 96 Conference Proceedings, pages 303–312,1996.[9] K.J. Dana, B. van Ginneken, S.K. Nayar, and J.J.Koenderink. Reflectance and Texture of Real-worldSurfaces. ACM Transactions on Graphics, 18(1):1–34,1999.[10] P. Debevec, T. Hawkins, C. Tchou, H-P Duiker,W. Sarokin, and M. Sagar. Acquiring the ReflectanceField of a Human Face. In SIGGRAPH 2000 ConferenceProceedings, pages 145–156, 2000.[11] P.E. Debevec, C.J. Taylor, and J. Malik. Modeling andRendering Architecture from Photographs: A HybridGeometry- and Image-Based Approach. In SIGGRAPH96 Conference Proceedings, pages 11–20, 1996.[12] T.A. Funkhouser, I. Carlbom, G. Pingali G. Elko,M. Sondhi, and J. West. Interactive Acoustic Modelingof Complex Environments. J. Acoust. Soc. Am.,105(2), 1999.[13] M. Garland and P.S. Heckbert. Surface SimplificationUsing Quadric Error Metrics. In SIGGRAPH 97 ConferenceProceedings, pages 209–216, 1997.95A162

[14] W.W. Gaver. Synthesizing Auditory Icons. In Proceedingsof the ACM INTERCHI 1993, pages 228–235,1993.[15] D.P. Greenberg, K.E. Torrance, P. Shirley, J. Arvo, J.A.Ferwerda, S. Pattanaik, E.P.F. Lafortune, B. Walter,S. Foo, and B. Trumbore. A Framework for RealisticImage Synthesis. In SIGGRAPH 97 Conference Proceedings,pages 477–494, 1997.[16] I. Guskov, K. Vidimce, W. Sweldens, and P. Schroder.Normal Meshes. In SIGGRAPH 2000 Conference Proceedings,pages 95–102, 2000.[17] H. Hoppe, T. DeRose, T. Duchamp, M. Halstead,H. Jin, J. McDonald, J. Schweitzer, and W. Stuetzle.Piecewise Smooth Surface Reconstruction. InSIGGRAPH 94 Conference Proceedings, pages 295–302,1994.[18] D.L. James and D.K. Pai. ArtDefo, Accurate RealTime Deformable Objects. In SIGGRAPH 99 ConferenceProceedings, pages 65–72, 1999.[19] D.L. James and D.K. Pai. A Unified Treatment ofElastostatic Contact Simulation for Real Time Haptics.Haptics-e, The Electronic Journal of Haptics Research(www.haptics-e.org), 2001. (To appear).[20] J. Lang and D. K. Pai. Estimation of Elastic Constantsfrom 3D Range-Flow. In Proceedings of the Third InternationalConference on 3-D Digital Imaging and Modeling,2001.[21] A. Lee, H. Moreton, and H. Hoppe. Displaced SubdivisionSurfaces. In SIGGRAPH 2000 Conference Proceedings,pages 85–94, 2000.[22] A. Lee, W. Sweldens, P. Schröder, L. Cowsar, andD. Dobkin. MAPS: Multiresolution Adaptive Parameterizationof Surfaces. In SIGGRAPH 98 ConferenceProceedings, pages 95–104, 1998.[23] M. Levoy, et al. The Digital Michelangelo Project: 3DScanning of Large Statues. In SIGGRAPH 2000 ConferenceProceedings, pages 131–144, 2000.[24] D. K. Pai, J. Lang, J. E. Lloyd, and J. L. Richmond.Reality-based Modeling with ACME: A Progress Report.In Proceedings of the Intl. Symp. on ExperimentalRobotics, 2000.[25] D. K. Pai, J. Lang, J. E. Lloyd, and R. J. Woodham.ACME, A Telerobotic Active Measurement Facility. InProceedings of the Sixth Intl. Symp. on ExperimentalRobotics, 1999.[26] Z. Popovic and A. Witkin. Physically Based MotionTransformation. In SIGGRAPH 99 Conference Proceedings,pages 11–20, 1999.[27] C. Rocchini, P. Cignoni, C.Montani, and P. Scopigno.Multiple Textures Stitching and Blending on 3D Objects.In 10th Eurographics Workshop on Rendering,pages 173–180, Granada, Spain, 1999.[30] P.J. Rousseeuw and K. Van Driessen. Computing LTSRegression for Large Data Sets. Technical report, Universityof Antwerp, 1999.[31] H. Rushmeier, F. Bernardini, J. Mittleman, andG. Taubin. Acquiring Input for Rendering at AppropriateLevels of Detail: Digitizing a Pietà. EurographicsRendering Workshop 1998, pages 81–92, 1998.[32] D.C. Ruspini, K. Kolarov, and O. Khatib. The HapticDisplay of Complex Graphical Environments. InSIGGRAPH 97 Conference Proceedings, pages 345–352,1997.[33] Y. Sato, M.D. Wheeler, and K. Ikeuchi. Object Shapeand Reflectance Modeling from Observation. In SIG-GRAPH 97 Conference Proceedings, pages 379–388,1997.[34] H. Spies, B. Jähne, and J.L. Barron. Dense Range Flowfrom Depth and Intensity Data. In International Conferenceon Pattern Recognition, pages 131–134, 2000.[35] K. Steiglitz. A Digital Signal Processing Primer withApplications to Digital Audio and Computer Music.Addison-Wesley, New York, 1996.[36] K. Steiglitz and L.E. McBride. A Technique for theIdentification of Linear System. IEEE Trans. AutomaticControl, AC-10:461–464, 1965.[37] D. Terzopoulos, J. Platt, A. Barr, and K. Fleischer.Elastically Deformable Models. In Computer Graphics(SIGGRAPH 87 Proceedings), volume 21, pages 205–214, 1987.[38] T.R. Thomas. Rough Surfaces. Imperial College Press,London, second edition, 1999.[39] K. van den Doel, P.G. Kry, and D.K. Pai. FoleyAutomatic:Physically-based Sound Effects for InteractiveSimulations and Animations. In SIGGRAPH 2001 ConferenceProceedings, 2001.[40] K. van den Doel and D.K. Pai. The Sounds of PhysicalShapes. Presence, 7(4):382–395, 1998.[41] S. Vedula, S. Baker, P. Rander, R. Collins, andT. Kanade. Three-dimensional Scene Flow. In InternationalConference on Computer Vision, pages 722–729,1999.[42] M. Yamamoto, P. Boulanger, J.-A. Beraldin, and M. Rioux.Direct Estimation of Range Flow on DeformableShape from a Video Rate Range Camera. PAMI,15(1):82–89, 1993.[43] Y. Zhang and C. Kambhamettu. Integrated 3D SceneFlow and Structure Recovery from Multiview Image Sequences.In Computer Vision and Pattern Recognition,volume 2, pages 674–681, 2000.[28] G. Roth and E. Wibowoo. An Efficient VolumetricMethod for Building Closed Triangular Meshes from 3-D Image and Point Data. In Proc. Graphics Interface,pages 173–180, 1997.[29] P.J. Rousseeuw. Least Median of Squares Regression.Journal of the American Statistical Association,79(388):871–880, 1984.96A163

The AHI: An Audio And HapticInterface For Contact InteractionsDerek DiFilippo and Dinesh K. PaiDepartment of Computer ScienceUniversity of British Columbia2366 Main Mall.Vancouver, BC V6T 1Z4, CanadaTel: +1-604-822-6625{difilip | pai}@cs.ubc.caABSTRACTWe have implemented a computer interface that renderssynchronized auditory and haptic stimuli with very low(0.5ms) latency. The audio and haptic interface (AHI)includes a Pantograph haptic device that reads positioninput from a user and renders force output based on thisinput. We synthesize audio by convolving the force profilegenerated by user interaction with the impulse response ofthe virtual surface. Auditory and haptic modes are tightlycoupled because we produce both stimuli from the sameforce profile. We have conducted a user study with theAHI to verify that the 0.5ms system latency lies below theperceptual threshold for detecting separation betweenauditory and haptic contact events. We discuss futureapplications of the AHI for further perceptual studies andfor synthesizing continuous contact interactions in virtualenvironments.KEYWORDS: User Interface, Haptics, Audio, Multimodal,Latency, Synchronizationlimit their ability to navigate and control their environment.An effective computer interface for interacting with asimulated environment would allow one to tap and scrapeon virtual objects in the same way one can tap and scrapeon real objects. In addition to providing visual feedback,the interface would also create realistic auditory and hapticcues. These cues would be synchronized so that theyappear perceptually simultaneous. They would also beperceptually similar -- a rough surface would both soundand feel rough. This type of interface could improve theamount of control a user could exert on their virtualenvironment and also increase the overall aestheticexperience of using the interface. For example, a systemdesigner might wish to use this interface to representremote objects to discerning user groups varying fromNASA scientists to e-commerce customers. In this paper,we present an experimental audio and haptic interface(AHI) for displaying sounds and forces with low latencyand sufficient realism for interactive applications.INTRODUCTIONWhen two objects collide, we have a contact interaction.Everyday lives are full of them: pulling a coffee cup acrossa table, tapping our fingers on a computer keyboard, etc.These contact interactions can produce characteristicsounds and forces that communicate information about ourrelationship with rigid objects and the surroundingenvironment. By using our ears and hands we can tell if thecoffee cup was placed safely on the table or if the table ismade of glass or wood, for example. Depriving someone ofthis sensory feedback from his or her interactions couldFigure 1: The AHI in a typical configuration.Figure 1 shows the AHI in a typical configuration. Theuser grips the handle with their right hand and moves it inA164

the plane as they would move a computer mouse. The leftside of the monitor screen shows a Java graphical userinterface for loading sound models and controllinginteraction parameters and the right side shows a graphicalwindow for viewing haptic forces and audio signals in realtime.The novelty of the AHI lies in the tight synchronization ofthe auditory mode and the haptic mode. User interactionwith the simulated environment generates contact forces.These forces are rendered to the hand by a haptic forcefeedbackdevice, and to the ear as contact sounds. This ismore than synchronizing two separate events. Rather thantriggering a pre-recorded audio sample or tone, the audioand the haptics change together when the user appliesdifferent forces to the object.Building a device to a certain technical specificationprovides no guarantee for how a user will perceive theeffect of the device. However, an over-specified device canbe used to help establish lower bounds for humanperception that give system builders a reliable design target.An analogy would be to the design of computer monitorhardware to support refresh rates of 60Hz. This refresh rateis well known to be sufficient for comfortable viewing andsufficient to simulate continuous motion. Our goal is to usethe AHI to help establish similar perceptual tolerances forsynchronized audio and haptic contact interactions.The first part of this paper describes the AHI hardware foruser input, models for calculating haptic and audio contactforces, and the control structure for rendering these forces.The second part of this paper presents experimental resultsthat suggest the 0.5ms system latency of the AHI lies belowthe perceptual tolerance for detecting synchronizationbetween auditory and haptic contact events. Establishingthat our interface works below perceptual tolerance enablesus to use it for psychophysical experiments for multimodalperception.HARDWAREUser input to the AHI comes from a 3 degree of freedom(DOF) Pantograph device (Figure 2). The 5-bar mechanismis based on a design by Hayward [14] but extended to 3DOF to our specification. It reads 3 DOF of position as userinput, and renders 3 DOF of forces as output. The user canmove the handle in the plane as well as rotate the handle.There are two large Maxon motors attached to the base ofthe Pantograph which apply forces on the handle via the 5-bar linkage. A small motor in the handle can exert a torqueon the handle as well. The device, therefore, is complete forrigid motions in the plane, i.e., it can render the forces andtorque due to any contact with a rigid body attached to thehandle in a planar virtual world (‘‘flatland’’). We do notcurrently use the third rotational DOF for our work with theAHI.Figure 2: The Pantograph haptic device.A dedicated motion control board (MC8, PrecisionMicrodynamics), hosted in a PC running Windows NT,controls the AHI. The board has 14 bit analog to digitalconverters (ADCs) for reading the potentiometers attachedto the base motors as well as quadrature decoders forreading the optical encoder which measures the handlerotation.A SHARC DSP (Analog Devices 21061 running at40MHz) on the MC8 synthesizes sounds and forces.Output voltages for controlling the Pantograph motors andfor rendering audio are sent out through 14 bit digital toanalog converters (DACs). The audio waveforms can beinput directly to a sound system; in our set up they are sentto the soundcard of the computer for ease of capture andplayback.By using this specialized hardware we bypass thecomplications that arise from balancing the needs of realtime,deterministic response and ease of access from userlevelsoftware on a widely available operating system suchas NT. The AHI control code is compiled for the DSP andhas exclusive control over its resources. This allows us toprecisely time our control algorithms as well as accuratelydiagnose inefficiencies and bugs. In particular, the overallsystem latency is 0.5ms for synchronized changes in audioand haptics.AUDIO SYNTHESISWe wish to simulate the audio response of everyday objectsmade out of wood, metal, ceramic, etc. Contact with theseobjects can be characterized by impulsive excitation ofrelatively few exponentially decaying, weakly coupledsinusoidal modes. Modal synthesis and impulse generationtechniques have been developed for these types ofpercussive sounds [2]. We use the modal audio synthesisalgorithm described in [20]. This algorithm is based onvibration dynamics and can simulate effects of shape,location of contact, material, and contact force. Modelparameters are determined by solving a partial differentialequation, or by fitting the model to empirical data [15].A165

The sound model assumes that the surface deviation yobeys a wave equation. We add a material-dependentdecay coefficient to the wave equation to damp the sounds.The exponential damping factor d = fπtan(φ) depends onthe frequency f and internal friction φ of the material, andcauses higher frequencies to decay more rapidly. Theinternal friction parameter is material dependent andapproximately invariant over object shape. Equation 1represents the impulse response of a general object at aparticular point as a sum of damped sinusoids.only knows about joint angles. We need a forwardkinematic mapping that gives the xy-position of the handleas a function of base joint angles, as well as a differentialkinematic mapping that gives the base joint torques as afunction of applied force to the handle.∑ ∞ −dnty(t)= anesin( ωnt)(1)n=1The sound model of an object consists of a list ofamplitudes a n and complex frequencies Ω n = ω n + id n .Equation 2 shows how one complex frequency iscomputed. At time 0, the signal is the product of thefrequency-amplitude a, and the contact force F(0). At eachsuccessive time step (determined by the samplingfrequency Fs), the signal is the sum of a decayed version ofthe previous signal plus a new product of amplitude andcontact force. The model responds linearly to input forceF(k). Once we have the model parameters, all we need tobegin synthesizing sounds is a series of contact forces toplug into the right-hand side of the recursion. The outputsignal at time k is Re(Σy n (k)), with the sum taken over allcomputed frequencies.y (0) = any ( k)= ennF(0)Ω (2)niFsy ( k −1)+ annF(k)This synthesis algorithm has two benefits. First, it is linear.The computed audio is the discrete convolution of the forcehistory with the impulse response. There is a naturalrelationship between the input forces and the output signal;this relationship would not be as straightforward if we usedsample-based synthesis. The linearity also makes itefficient. With a basis change, an audio signal can becomputed with 2 multiplications and 3 additions percomplex frequency, per sample. Second, the audio qualitycan degrade gracefully. If DSP time is running short wecan compute fewer frequencies, resulting in a smooth lossof detail rather than sudden audio dropouts.HAPTIC FORCE SYNTHESISAs the user moves the Pantograph handle we need tocompute the contact forces resulting from theseinteractions, and then render them as forces on the handleof the Pantograph by exerting torques on its base joints.These computations take place in two coordinate frames.One is the world frame of xy-coordinates and the other isthe Pantograph frame of joint angles. The simulatedenvironment uses the world frame, but the control codeFigure 3: Pantograph kinematics.For the forward kinematic mapping, we specify the basejoint of motor 1 as the origin of the world frame. There is ageometric constraint that allows us to compute the positionof the handle: the vector pointing from elbow 1 to elbow 2(e 21 = e 2 -e 1 ) in the world frame is always perpendicular tovector pointing to the handle from the midpoint of e 21 . If q 1and q 2 are the base joint angles, then the elbows becomee 1 = (Lcos(q 1 ), Lsin(q 1 )) and e 2 = (Lcos(q 2 ), Lsin(q 2 )) whereL is the length of the proximal arms, and S is the separationbetween the two base joints. Setting e ⊥21 as the vectorpointing from the midpoint of e 21 to the handle h, we haveh(q) = e 1 + 0.5e 21 + e ⊥ 21 . This expression for h in terms ofjoint angles q has a simple geometric interpretation, asshown in Figure 3.Once we have the handle coordinates, and compute acontact force F, we need to transform this force into basejoint torques τ for rendering. The Jacobian J=∂h(q)/∂q ofthe forward kinematic mapping relates forces to torques byJ T F=τ. The details of constructing the Jacobian for thePantograph are quite general and are covered in basicrobotics texts [11]. In our particular implementation, wecan avoid the expense of computing the partials of h(q) byexploiting the structure of the Jacobian. Details areremoved here for the sake of brevity.NORMAL FORCESFor interactions normal to the surface of a plane, aspring/damper/impulse combination constrains the user tothe surface by applying a penalty force. If the normaldisplacement past the surface is x n , and the current normalvelocity is v n , then the haptic constraint force is F = Kx n +Dv n where K and D are spring and damping constants. For10ms after a new contact, we add a unilateral impulse Pv nA166

to the spring/damper combination. This technique isknown to increase the perception of haptic stiffness withoutintroducing closed-loop instabilities that can occur withlarge spring coefficients [16].force discontinuously drops to zero, and (2) high frequencyposition jitter.TANGENTIAL FORCESFor interactions tangential to the surface of a plane, wehave implemented a stick-slip friction model to provideforce feedback. Stick-slip models exhibit two states:slipping and sticking. Specifying a stick-slip modelrequires defining state transition conditions. In general, avirtual proxy point connects the real contact point to thesurface. In the sticking state, the real contact pointseparates from the proxy and frictional force opposesfurther motion proportional to the separation. When acontact point is in a sliding state, the virtual proxy slidesalong with it.Hayward’s stick-slip model only uses displacements todetermine state transitions [6]. If we define z = x k - x proxy asthe displacement between the real contact point and theproxy and z max as the maximum displacement then theupdate for the next proxy point is x proxy = x k ± z max if α(z)|z|> 1 (slipping), or x proxy = x proxy + |x k – x k-1 |α(z)|z| otherwise(sticking). Once the displacement between the proxy andreal contact point passes a maximum the contact becomesfully tense and enters the slipping state. The proxy pointand the real contact point move together, separated by z max .For displacements less than the maximum the proxy pointdoes not move much; this is the sticking state. The nonlinearadhesion map α(z) allows the proxy point to creepbetween these two regimes.We have only selected two of many possible haptic forcemodels that could be used by the AHI to represent surfaceproperties. Stochastic models for haptic textures based onbased on filtered noise and fractional Brownian motion arewell known in the haptics community [5,18]. For example,perturbing the normal force by sampling from a Gaussiandistribution generates haptic sandpaper. Friction modelshave a long history [1]. The algorithms and techniques weimplement for planar normal and tangential forces areintended to be incorporated with more sophisticatedapplications that manage their own collision detectionbetween complex polygonal geometries. After collisiondetection, these applications could use any suitable modelfor computing local normal and tangential forces.AUDIO FORCE SYNTHESISNaively using the raw normal forces to synthesize audioproduces unsatisfying results. There are two properties ofour synthesized normal forces that cause trouble. Thissection will describe how we filter out these two propertiesfrom the haptic force. The filtered result is the audio forcethat we convolve with the stored impulse response inEquation 2. The two properties of the haptic force that wewish to filter are as follows: (1) a spurious impulse resultswhen the user breaks contact with the surface and the hapticFigure 4(a): Idealized haptic and audio forceprofiles.Figure 4(a) plots an idealized haptic contact force. At 30msthe user comes into contact with the surface and stays incontact for another 30ms. Convolving this square waveprofile with the impulse response of the surface willproduce a spurious second ‘‘hit’’ when the user breakscontact. We introduce an attenuation constant β to allowthe audio force to smoothly move to zero during sustainedcontact. If t is the elapsed time since contact, then thecurrent audio force is the current haptic force attenuated byβ t . We have found that attenuating the audio force starting10ms after a new contact with β = 0.85 (halflife of 5ms)produces good results. Waiting 10ms before decayingimproves the quality of impulsive contacts, whereasdecaying the audio force immediately upon contactexcessively reduces the overall amplitude and dynamicrange of the resulting audio signal.Figure 4(b): A captured audio force profile.Haptic instabilities and signal noise generate sustained lowamplitude, high frequency jitter in the position readings.These high frequencies are passed into the haptic forceprofile by the linear spring constant. Without filtering, thisA167

noise becomes audible as crackling and popping while theuser maintains a static contact with the surface. This lowamplitude noise can be removed by truncation. Typically,we remove the 8 lowest order bits. Figure 4(b) plots atypical audio force profile. (The signal is not perfectlyconstant during each millisecond interval because it wascaptured as the input signal to a soundcard.) Using discreteoptical encoders instead of potentiometers to read jointangles removes the need to truncate the position signals –the finite resolution of the encoders effectively truncates thesignal for us.REAL-TIME SIMULATIONThe basic control structure for the AHI real-time synthesisand simulation is interrupt-driven. There is a hapticinterrupt service routine (HISR) that generates haptic andaudio forces and an audio interrupt service routine (AISR)that convolves the audio force with the impulse response ofthe modelled object. Using these two separate interrupts wecan synthesize the audio signal at a much higher rate thanwe generate haptic feedback. This section will describe thetwo interrupt routines shown in Figure 5.The AISR and all DAC/ADC latches are synchronized totrigger at the audio control rate by using a programmableinterval timer that counts at half the bus clock rate of 8.33MHz. The AISR reads the Pantograph joint angles fromthe ADCs and stores them in an array that contains ahistory of joint angle readings. Converting the DAC inputto an equivalent floating point number requires 1comparison, 2 multiplications, and 2 additions. The currentaudio force is then clipped to lie between 0.0 and 1.0 andtruncated to remove low amplitude noise. This requires 2comparisons and 2 multiplications. A discrete convolutionstep using this filtered audio force F(k) produces the outputaudio signal y n (k). This signal is placed in the DAC out.Computing the audio signal requires 2 multiplications and 3additions per complex frequency, per sample. If the DSP isshort on cycles, we can decrease the number of activefrequencies. In our current scheme this isn't necessary --there are no other competing processes for DSP time. Oncea number of complex frequencies are selected the totalamount of processing time is fixed and does not need to beadaptively adjusted. This would change if the DSP wasalso managing a complicated environment with graphics,collision detection, and rigid body dynamics.Haptic interrupts trigger at an integer fraction of the audiocontrol rate. The current joint angles are the mean of thearray of joint angles captured during the AISR. From thesefiltered values, we use the forward kinematics of thePantograph to compute the handle position. Since we onlyconsider interactions with a plane, determining contactbetween the handle and the plane takes a sign check. IfFigure 5: Flow of control for real-time synthesis and simulation.A168

there is contact, we compute the resulting normal forceusing a spring/damper/impulse model and a tangential forceusing a stick-slip friction model. Updating the currentJacobian takes 22 multiplications, 4 trigonometric calls, and2 square roots. The Jacobian of the Pantograph translatesthe haptic force into motor torques. The voltages togenerate these torques are written to the DACs. If there hasbeen contact for (10+t) milliseconds, then β t times thenormal force plus the tangential force becomes the currentaudio force. The HISR writes the current audio force to aglobal variable shared with the AISR.CONTINUOUS CONTACT INTERACTIONSWe have experimented with some examples using the basiccontrol structure just described to demonstrate how the AHIcan generate continuous audio and haptic contactinteractions. In the first example, the user scrapes the AHIhandle across a sinusoidally modulated surface profile. Inthe second example, the user slides the handle across asurface with stick-slip friction. In both examples, weconvolve the resulting audio force with the impulseresponse of a brass vase acquired from the University ofBritish Columbia Active Measurement facility [15]. Theseexamples have been informally tested in our laboratory.Haptic interrupts trigger at 1kHz and audio interrupts at20kHz. Thus, there is a 1ms latency for changes in forceand audio. Informally, the auditory and haptic stimuli areperceptually simultaneous.scraping along the ribbed surface of the vase with the tip ofa pen.Figure 7 shows captured audio signal and audio forcemagnitudes when interacting with the AHI and Hayward’sstick-slip friction model. Audio force decay was enabledfor this captured signal, but only applied to the normalforce component. The user interaction in this example wasto slide the handle tangentially across a flat plane. A forcehysteresis is apparent. The force increases as thedisplacement between the real and proxy contact pointincreases (sticking phase) and then discontinuously drops tozero during the slip phase. These discontinuities in theaudio force create impulses in the audio signal. We used asimilar model of a brass vase in this example to the one inFigure 6.Figure 7: Output audio signal and audio forcemagnitude for interacting with the AHI andHayward’s stick-slip friction model.Figure 6: Output audio signal and audio forcemagnitude.Figure 6 plots a captured audio force and the convolution ofthis force with the measured impulse response of the vase.We compute 20 modes for each audio sample. The userinteraction in this example was five single strikes ofincreasing force normal to the surface, then tangentialmotion across the surface. The middle two bursts areslower scrapes back and forth, and the final two bursts arefaster scrapes. The auditory signals produced in this fashionare satisfying. ‘‘Zipper’’ audio effects can be created byrapidly scraping on the surface. These synthesized audiosignals compared favorably to live signals of tapping andUSER STUDYThe remainder of the paper will describe a user study weconducted with the AHI. In this user study we tested thehypothesis that a 2ms latency lies below the perceptualtolerance for detecting synchronization between auditoryand haptic contact events. We selected 2ms because it islarger than our 0.5 system latency, but a small enoughinterval to establish a lower bound. Briefly, a subjecttapped on a virtual wall and received appropriate auditoryand haptic stimuli, except that one of the two was delayedby 2ms. We tested the hypothesis that all subjects wouldperform at chance when asked to choose which stimuluscame first.ParticipantsTwelve members of our department (2 females, 10 males)participated in the user study. Their mean age was 32years, with a minimum age of 21 and a maximum age of55. There was one left-handed participant. All twelvereported normal hearing and touch. The participants werenot paid for their time.A169

Apparatus and StimuliWith a few software modifications, we used the AHI asdescribed in the first half of this paper. The stimuliconsisted of 24 contact events where the audio signalpreceded the haptic by 2ms, and 24 contact events wherethe haptic signal preceded the audio signal by 2ms. Thehaptic control rate was 2kHz and the audio control rate was20kHz, resulting in a 0.5ms system latency. We createdprecedence by delaying one of the two signals with a shortcircular buffer. Delaying the haptic signal did not causeany instability. We disabled tangential forces.A vertical ‘‘wall’’ was positioned in the right half of theworkspace. One set of 48 random locations of the wallwithin ±1cm were generated and used in the same order forall subjects. Haptic force-feedback was provided using thespring/damper/impulse combination. For audio, strikingthe wall was treated as striking a single point on an idealbar, clamped at both ends. The contact point was at 0.61 ofthe length of the bar. For a given fundamental frequency,this simple geometry has an analytical solution for thefrequencies and relative amplitudes of the higher partials.We used fundamental frequencies ω 1 of 1000Hz (High) and316Hz (Low) and four higher partials.As shown in equation 1, the decay of these frequencies aredetermined by a damping coefficient d = fπ tan(φ). We usedtwo values of τ d = 1/(π tan(φ)) that correspond to slowdecay and fast decay: 300 (Fast), and 3 (Slow). Theseparticular auditory stimuli were selected because they are asubset of those used for a study that connects variation ofthe coefficient τ d to auditory material perception [7].Experimental DesignThe experiment used a two-alternative forced choicedesign. The subjects were asked to decide whether theaudio signal preceded the haptic signal, or the haptic signalpreceded the audio signal.A three within-subject design was used with audio/hapticprecedence, frequency, and damping as the within-subjectvariables. In total, there were 8 different stimuli presentedto the user. Audio precedence coupled with one of fourfrequency/damping combinations (High + Fast, Low + Fast,High + Slow, Low + Slow), and haptic precedence coupledwith the same four sound combinations. Six of each ofthese 8 different types were permuted once and used in thesame order for all subjects for a total of 48 stimuli.Experimental ProcedureSubjects sat on a chair with the Pantograph on a desk infront of them. The Pantograph base was affixed to thedesktop with a rubber sheet to minimize sliding androtating. Subjects wore closed headphones (AKG K-240)for the audio signals and to minimize external soundsincluding those from the Pantograph device. We told thesubjects that they would be striking a virtual wall, and thatthis would produce both haptic forces and audio signals.They were told that, in each case, one of the stimulipreceded the other. We demonstrated how to hold thehandle of the pantograph and how to make an impulsivestrike to the wall. Then, the subjects were allowed topractice a few strikes with the headphones on. Finally, theexperiment began.Subjects were not told that there were equal numbers ofstimuli types, nor were they told the number of repetitionsin the experiment. No requirement on striking force wassuggested -- subjects were free to strike the wall as hard orsoft as they wished, as long as it was a single strike. Therewere no visual cues; however, the subjects were notblindfolded. After being read the the subjects were allowedto ask questions about the purpose of the experiment. Ifthey expressed some concerns about their ability todiscriminate between the two alternatives they were toldthat the discrimination task was designed to be difficult andto expect some ambiguity.RESULTSTable 1 shows the mean number of correct responses, alongwith the maximum, minimum, and standard deviation ofcorrect responses out of 48. Table 1 also shows the numberof audio responses selected. Four different subjects wereresponsible for each of the maxima and minima.# Of CorrectResponses# Of Audio ResponsesSelectedMean 21.08 23.58Max 26 31Min 17 16Std 2.57 4.52Table 1: Number of correct responses, and numberof audio responses selected out of 48, for all 12subjects.We test the null hypothesis that the subjects perform at thechance level (each response is a pure guess), for each of the12 subjects. By hypothesis, the mean number of correctresponses µ = 24 and the standard deviation σ = 3.45.Using the normal approximation to the binomialdistribution, we see that we can reject the hypothesis with atwo-tailed test at the significance level α = 0.05 only if thesample mean is outside the interval µ ± 1.96σ = [17.21,30.78]. Except for the lone subject with 17 correctresponses, we cannot reject the hypothesis that the subjectsare performing at chance. We note that a one-tailed testmay be more appropriate since we want to know if thesubjects can detect the precedence better than chance. Witha one tailed test, we can not reject the hypothesis for any ofthe subjects.DISCUSSIONThe results indicate that 2ms is a valid lower bound for theperceptual tolerance of synchronization latency of a contactinterface like the AHI.A170

Other data suggests that the rate of decay of the auditorystimulus is a factor in the user responses, independent ofthe actual order of stimulus presentation. Table 2 containsthe six most extreme number of correct responses, listedacross stimuli. Sounds that decay more slowly areperceived as preceding the haptic stimulus and sounds thatdecay quickly are perceived as lagging the haptic stimulus,independent of the actual order of stimulus presentation. Anincrease in the audio decay rate (sounds decaying morequickly) reduces the total energy and total duration of theaudio signal. It is possible that the subjects are using decayrate or total duration or total energy as a criterion for theirresponse. The subject could be choosing the ‘‘loudest’’signal (in terms of duration or energy) as the precedentstimulus. More studies are required to describe (andeventually understand) this effect.Stimulus # # Of Correct Stimulus TypeResponses4 1 Audio + High + Fast6 2 Haptic + High + Slow11 10 Haptic + High + Fast30 10 Haptic + Low + Fast33 1 Haptic + Low + Slow45 2 Haptic + Low + SlowTable 2: The six most extreme values for number ofcorrect responses, listed across stimuli. The number ofcorrect responses is out of 12. Stimulus type is listed inorder of precedence, frequency, and decay.FUTURE WORKIn this last section we consider further opportunities forusing the AHI in perceptual studies of integrated audio andhaptics, specifically for exploring multimodal texture andfriction. One of the basic questions about cross-modalsynchronization has been addressed recently, but manyquestions about similarity and synchronization betweendynamic vibration stimuli remain.Levitin, et al., have helped to identify the perceptualtolerance for synchronization between auditory and hapticcontact events [10]. Our original plan was to conduct avery similar study. Nevertheless, the AHI is well suited tohelp establish similar perceptual tolerances for continuousaudio and haptic contact interactions such as scraping andsliding over textured surfaces.In the Levitin study, subjects manipulated a baton. Theywould strike a horizontal surface (containing a capacitor)with this baton. By tracking position, velocity, andacceleration, the experimenters could predict the time ofactual impact. Using these predictions, a digitized sampleof a stick striking a drum was played back to the subject atrandom temporal offsets varying between -200ms and200ms. Subjects were asked to judge whether the batonstrike and the audio sample occurred at the same ordifferent times. The threshold where subjects consideredthe stimuli to be synchronous 75% of the timecorresponded to -19 and 38ms; that is, the interval betweenthe audio preceding the haptics by 19ms, and the audiolagging the haptics by 38ms. When adjusted for responsebias (using confidence ratings) the corrected thresholds fordetecting synchrony are -25ms and 66ms. Levitin’s practicalmotivation for this study was to determine an upper limitfor reliable perceptual synchronization that gives systemdesigners a well-defined performance target. The 66ms75% performance threshold is higher than what weexpected partially based on our own experience with theAHI.There may be an important difference in synchronizationtolerances between active haptic devices with motors suchas the AHI and passive devices such as the baton used inthe Levitin study. The AHI’s motors do not operate quietly.Motor torques excite structural vibrations which producesounds. It is known that the time resolution for successiveaudio clicks is on the order of 2ms and that this value islargely independent of frequency [13]. An intra-modaljudgement would use both audio signals (one from thespeakers, one from the motors) over the cross-modaljudgement that compares the audio signal to the hapticsignal. If this intra-modal judgement dominates the crossmodaljudgements then it will be necessary to decrease theasynchrony to well below Levitin’s reported figure.Identifying and controlling asynchronies for active deviceslike the AHI does not directly address cross-modalperception, but given the preponderance of active hapticdevices entering the market it would still be a valuableresult to derive.Our current simulation generates audio forces from hapticforces normal and tangential to a locally flat patch. Largescale surface features can consist of a collection ofpolygons which use our flat patch algorithms after collisiondetection. The spring/damper/impulse penalty methodeffectively parameterizes the normal component as surfacehardness. Limited versions of small-scale surface features⎯ friction and texture ⎯ have been implemented anddescribed in this paper. We want simple and effectiveparameterizations of the force components as friction androughness variables that remain relevant for auditoryperception.Hayward’s stick-slip model has been applied to the realtimephysical modeling of violin bow-string interactions[17]. Bow-string interactions are some of the oldest studiedexamples of stick-slip friction. A good test for addingfriction to our audio synthesis routine would be to simulatebowing a violin string and forcing it to resonate. Our AHIsimulations of this sort of behaviour are not convincing yet⎯ the audio signal in Figure 7 sounds more like a series ofimpacts than like the squeaking we associate with stick-slipphenomena. We might also use this model for synthesizingrough textures by imposing Gaussian noise perturbation oneither the separation between the proxy and real contact, oron the spring coefficient that produces friction forces. OurA171

experience is that we will need another stage of forceprefiltering to control the auditory roughness signal.Perhaps we could leverage work done in computer graphicson synthesizing fractional Brownian motion and fractals togain more control over our signals [4].Previous studies on the influence of auditory stimuli onhaptic perception have used audio samples or tonestriggered by contact events. The study by Miner, et al, usedpitched tones and attack envelopes to simulate hard and softsounds [12]. They found that ‘‘the auditory stimulus didnot significantly influence the haptic perception’’. The studyby DiFranco, et al, triggered audio samples of contactevents they recorded by hand [3]. They found that ‘‘soundcues that are typically associated with tapping hardersurfaces were generally perceived as stiffer’’. Both of thesestudies focus on the perception of hardness, which is afunction of force on the user’s hand. These studies do notexplore the perception of surface roughness, which inaddition to being a function of force, can also be a nontrivialfunction of interaction speed and surface geometry.The spatial characteristics of the surface dominateroughness perception when using a bare finger. The speedof interaction in this case does not affect roughnessperception. However, dynamic vibration effects as afunction of speed are reported when interaction is mediatedby a rigid probe [9]. These effects are complex and notfully understood. Increasing speed tends to render surfacesas smoother; however, unlike perception with the barefinger, the current effect tended to reverse itself as theinterelement spacing increased.A texture model for any haptic interface (not just the AHI)could use these perceptual results to inform theirimplementation. However, because the AHI tightly couplesthe auditory stimulus with the underlying physical processof collision, a simple grid produces haptic and auditorytextures that vary as a function of both force and speed(Figure 6). Previous studies on perceiving auditory andhaptic textures with the bare hand suggest that the subjectwill use the haptic texture before the auditory texture forroughness discrimination [8]. The similar andsynchronized stimuli rendered by the AHI could be used toextend these results for multimodal vibration effects when arigid probe mediates interaction.Devising and verifying new multimodal friction and texturemodels will require several psychophysical studies. Webelieve the AHI in its current state provides an excellentplatform for experimenting with and understanding thesemodels.CONCLUSIONDesigning compelling simulated environments is the highlevelgoal of this research. The representation andrendering of contact interactions comprises an essentialcomponent of any such simulation. Atomically representingthe contact event as something that produces both soundand force helps integrate auditory and haptic stimuli. Webelieve this is a natural way to think of representing andsimulating contact. We have implemented a new interfacethat can render audio and haptic interaction atomically.Our experimental results suggest that the AHI’s overalllatency and synchronization between the auditory andhaptic modes lies below the perceptible threshold. In thefuture, we will use the AHI to help explore perceptualsynchronization and similarity between continuous auditoryand haptic contact interactions.ACKNOWLEDGMENTSThanks to Susan Lederman for valuable discussions.Thanks to Paul Kry for timely debugging wisdom. Thiswork was supported in part by grants from the IRIS NCE,and Natural Sciences and Engineering Research Council ofCanada. Derek DiFilippo is supported by an NSERCscholarship and in part by the Advanced Systems Instituteof British Columbia.REFERENCES1. Armstrong, B., Dupont P. and Canudas de Wit C. ASurvey of Models, Analysis Tools and CompensationMethods for the Control of Machines with Friction.Automatica. 30,7 (1994), 1083—1138.2. Cook, P.R. Physically Informed Sonic Modeling(PhISM): Synthesis of Percussive Sounds. ComputerMusic Journal. 21,3 (1997), 38—49.3. DiFranco, D.E. and Beauregard, G.L. and Srinivasan,M.A. The Effect of Auditory Cues on the HapticPerception of Stiffness in Virtual Environments. Proc.ASME Dynamic Systems and Control Division, (1997).4. Ebert, D.S., Musgrave, F.K., Peachey, D., Perlin, K.and Worley, K. Texturing and Modelling. APProfessional, New York NY, 1998.5. Fritz, J.P. and Barner K.E. Stochastic models for haptictexture. Proc. SPIE Int. Symp. on Intelligent Systemsand Advanced Manufacturing, Boston MA, November1996.6. Hayward, V. and Armstrong, B. A new computationalmodel of friction applied to haptic rendering.Experimental Robotics VI, Lecture Notes in Controland Information Sciences, Springer-Verlag, NY, Vol.250, 403—412.7. Klatzky, R.L., Pai, D.K. and Krotkov, E.P. HearingMaterial: Perception of Material from Contact Sounds.To appear in Presence, October 2000.8. Lederman, S.J. Auditory Texture Perception.Perception. 8 (1979), 93—103.A172

9. Lederman, S.J., Klatzky, R.L., Hamilton, C.L. andRamsay, G.I. Perceiving Surface Roughness via aRigid Probe: Effects of Exploration Speed and Modeof Touch. The Electronic Journal of Haptics Research,1, 1, (1999).10. Levitin, D.J., MacLean, K. and Mathews, M. ThePerception of Cross-Modal Simultaneity. To appear inInt. Journal of Computing Anticipatory Systems, 2000.11. Murray, R.M., Li, Z. and Sastry, S.S. A MathematicalIntroduction to Robotic Manipulation. CRC Press,Ann Arbor MI, 1994.12. Miner, N., Gillespie, B. and Caudell, T. Examining theInfluence of Audio and Visual Stimuli on a HapticDisplay. Proc. of the 1996 IMAGE Conf., Phoenix AZ,June 1996.13. Pierce, J. Hearing in Time and Space. Chapter 8 ofMusic, Cognition, and Computerized Sound. The MITPress, Cambridge MA, 1999.14. Ramstein, C. and Hayward, V. The Pantograph: a largeworkspace haptic device for a multi-modal Humancomputerinteraction. Conf. on Human Factors inComputing Systems ACM/SIGCHI, Boston MA, April1994.15. Richmond, J.L. and Pai, D. K. Active Measurement ofContact Sounds. Proc. of the 2000 IEEE Int. Conf. onRobotics and Automation, San Francisco, April 2000.16. Salcudean, S.E., and Vlaar, T. On the Emulation ofStiff Walls and Static Friction with a MagneticallyLevitated Input-Output Device. ASME J. DynamicSystems, Meas., Control. 119 (1997), 127—132.17. Serafin, S., Vergez, C. and Rodet, X. Friction andApplication to Real-time Physical Modeling of aViolin. Int. Computer Music Conf., Beijing, October1999.18. Siira J. and Pai D.K. Haptic Textures — A StochasticApproach. IEEE International Conference on Roboticsand Automation, Minneapolis MN, April 1996.19. Srinivasan, M.A. and Basdogan C., Haptics in VirtualEnvironments: Taxonomy, Research Status, andChallenges. Comput. & Graphics. 4, 21 (1997), 393—404.20. van den Doel, K. and Pai, D.K. The Sounds ofPhysical Shapes. Presence 7,4 (1998), 382—395.A173

Page 1/6 Version 1.1 04/20/05Haptics for Scientific VisualizationRussell M. Taylor IIUniversity of North Carolina at Chapel Hillhttp://www.cs.unc.edu/~taylorrThe visionThe use of haptics in scientific visualization has barely begun, yet it already shows great promise.While haptics may be useful for pure visualization tasks, its true power comes from its ability to allow theuser to simultaneously push on and be pushed by the computer. This allows direct and immediate controland sensing of parameters in a simulation.Somewhere in the not-too-distant future… your simulation has been running for days on thesupercomputer. You decided to step in and take a peek to make sure it is going in the right direction.Stepping into the VR station, you bring up a 3D, tracked view of isosurfaces on the data set. To yoursurprise, a bubble has formed in the wave front. You can't see anything obvious, so you use your forceprobe to feel the area around the bubble. Tiny vibrations at the site indicate instability in the simulation dueto the simulation grid spacing being too sparse. You can fix the instability by re-meshing, but the bubble isstill there. You need to push it back up to match the wave, but you don't want to add too much energy atany time step in order to avoid introducing more instability. By mapping the derivative of added energy toforce, you feel the amount you are adding and are able to smoothly pull the simulation back on track.These musings give glimpses of where haptic visualization may lead us. Let’s take a look as where weare and how we got here. Along the way, we’ll see concrete examples of the usefulness of haptics inscientific visualization.Haptic display history (highlights)The history of haptic display and the study of human haptic and tactile perception is wide and varied,and covered in other sections of this book. Presented here are some highlights of this history that dealclosely with the use of haptics for scientific visualization or form the basis for techniques that may beemployed for this purpose.The Sandpaper system for texture synthesisMargaret Minsky developed the Sandpaper system for synthesizing texture in a force-feedback displaysystem, culminating in her 1995 dissertation at MIT on the subject. (Minsky, Ouh-young et al. 1990;Minsky 1995) This system was a 2D force-feedback joystick that allowed users to feel around on 2Dtextures that were computed or read from images. The “texture” in this system included both large-scalesurface shape information and small-scale texture information. Lateral force was presented based on thelocal slope of the surface height map, with the joystick pushing in the direction that would be “down” onthe surface. The amount of force was greater when the surface was more steeply sloped. Even though onlylateral forces were presented, users perceived that they were moving a stylus up and down over bumps anddips in a surface.Of interest for scientific visualization, screen-based sliders (adjusting the viscosity or spatial frequencyof a computed texture, for example) could control Sandpaper’s texture parameters. If the value of theseparameters were mapped to spatially varying scalar or vector fields defined on a surface, the result wouldbe a texture field whose properties depended on (and displayed) the underlying data sets. This has thepotential to allow the display of multiple data sets on the same surface.The user studies performed with the Sandpaper system can inform the selection of mappings from datavalues to texture parameters. Minsky explored the perception of surface roughness and found that for thecase of small periodic ridges the roughness percept can be almost entirely predicted by the maximumlateral force encountered while feeling the simulation. She also proposed a framework for haptic modelsbased on both physically-based and perceptually-based representations of the haptic properties of objectsand situations. (Minsky 1995)A174

Page 2/6 Version 1.1 04/20/05Remote micro-machiningCollaboration between the University of Tokyo and George Washington University resulted in asystem that provided local visual, haptic and auditory presentation of the action of a remote milling tool.(Mitsuishi, Hatamura et al. 1993) Their goal was the creation of a teleoperation system for remote controlof a milling machine. Due to the latency of transmission and the small amount of available communicationbandwidth, they used an intermediate model to provide force and auditory feedback. Furthermore, theypointed out that at very small scales, friction, viscosity and static charge may play a much more importantrole than inertial forces, so direct mapping of forces may be misleading and some translation may berequired to allow “natural” operation by the user. This amounts to building a simulation of millingoperation that the user interacts with, and whose parameters are driven from the actual, remote millingoperation. Thus, their work gives an example of visualizing the behavior of a remote milling tool based ona local model.They performed averaging on the force signal to remove the strong 33.3 Hz component due to rotationof the cutting tip. They examined the offsets in the tool to determine whether chatter is occurring andsimulated chatter at the user's end when it occurred. Because they were using prediction to overcomelatency, and because prediction can produce incorrect motion, safeties were put in place on the device endto prevent over-force or other dangerous conditions at the tool end. When performing machiningoperations, the degrees of freedom of the tool were reduced relative to those of the user (the drill wouldonly go up and down, for example) in order to increase precision over that of the human motor system. Theuser could also specify start and endpoints for a milling trajectory and then have the tool follow a nearestneighborpath along this trajectory with speed controlled by the user.Tool rotation speed was encoded and displayed to the user as a sound whose tone varied to indicatespeed. They also encoded information in sound location, with sounds to the right meaning the tool wasmoving to the right. Discontinuous sound caught the user's attention and was used to emphasize rapidchanges in velocity, which might indicate dangerous conditions.Display of force fieldsAn early haptic feedback application developed at the University ofNorth Carolina at Chapel Hill allowed the user to feel the effects of a 2Dforce field on a simulated probe, and was used to teach students in anintroductory Physics course. (Brooks, Ouh-Young et al. 1990) Theseexperiments were performed using a 2D sliding-carriage device that usedpotentiometers for position and servomotors for force presentation.Experimental results showed that this feedback improved the understandingof field characteristics by students who were interested in the material.Students reported that using the haptic display dispelled previousmisconceptions. They had thought that the field of a (cylindrical) diodewould be greater near the plate than near the cathode, and they thought thegravitation vector in a 3-body field would always be directed at one of thebodies.Molecular modelingMing Ouh-young at UNC-CH designed and built a hapticfeedback system to simulate the interaction of a drug moleculewith its receptor site in a protein. (Brooks, Ouh-Young et al.1990; Ouh-young 1990) This system, called the Docker,computed the force and torque between the drug and proteindue to electrostatic charges and inter-atomic collisions. Theseforces were presented to the user, pulling the drug towardslocal energy minima. This task is very similar to that of other“lock and key” applications where the user moves one objectand senses collisions with other objects in the environmentThe system presented the force and torque vectors bothvisually and using haptic feedback. Experiment showed thatA175

Page 3/6 Version 1.1 04/20/05chemists could perform the rigid-body positioning task required to determine the lowest-energyconfiguration of the drug up to twice as quickly with haptic feedback turned on compared to using thevisual-only representations. (Ouh-young 1990) Scientists also reported that they felt like they had a betterunderstanding of how the drug fit into the receptor site when they were able to feel the forces.The Docker application, like other path-planning applications, required the presentation of both forceand torque to the user. Since the drug molecule was not a point probe, different portions of it could collidewith the protein at the same time. Extricating the drug from a collision sometimes required both translationand twisting. If the user was provided with only force (translation) information and no torque (twist)information, they could be led to move the drug in an improper direction.Haptic visualization for the blind (and the sighted)The Applied Science and Engineering Laboratories at the University of Delaware have been pursuinghaptic visualization in the context of providing visualization systems that are suitable for use by the blind orvisually impaired. (ASEL 1998) The haptic work was coordinated by Jason Fritz, who completed hismaster’s thesis on haptic rendering techniques for scientific visualization in 1996. (Fritz 1996) In it, hedescribes several results, some of which are listed here.Fritz found that the haptic equivalent to the grid lines on a 2D graph were very helpful in providingscale information and aid in navigation without being distracting. His implementation of this was toproduce parallel planes evenly spaced in the volume that felt like thin walls through which the probepenetrates while moving through the volume where data is displayed. Fritz also discusses using friction andtexture to make the simulated surface feel more realistic and to distinguish features in a data set. Hedescribes a stochastic model for texture generation that can be used to create information-rich haptictextures for surfaces. (Fritz and Barner 1996)Haptic visualization circa 1998Several groups around the world are actively pursuing the haptic presentation of scientific data. Thesegroups often include haptic feedback into systems that already use graphical or auditory data presentation.While care must be taken to avoid the effects of conflicting cues ((Srinivasan, Beauregard et al. 1996;DiFranco, Beauregard et al. 1997)), this can form a powerful combination. Haptics forms the only bidirectionalchannel between the user and the computer; each can push on the other. This allows the user tosimultaneously sense the state of a system and control its parameters. Presented here is the work and resultsof some of these groups.Volume visualizationIwata and Noma at the University of Tsukuba built a haptic feedback force/torque sensor and HMDsystem to display volume data. (Iwata and Noma 1993) The system displays scalar data (the densityfunction) as either torque about Z depending on density, force depending on density gradient, or bothcombined. They found that position error was reduced by a factor of two as compared to visual feedbackalone when either or both of these forces were enabled. They describe one possibility for viewing multiparameterdata (fluid flow) by mapping flow velocity into force and one component of vorticity into torquearound the direction of flow.Avila and Sobierajski have developed a system that displaysvolume data sets both visually and haptically, and allows the userto modify the data sets. (Avila and Sobierajski 1996) Their systemhas been applied to medical visualization, art and scientificvisualization. They have shown how the visual exploration of acomplex 3D data set, such as this confocal scan of a lateralgeniculate nucleus (LGN) neuron, can be enhanced through theuse of haptics. In this example a user is able to feel the structure ofthe cell and follow dendrites through complicated winding paths.A gentle attracting force was used to follow the dendrites sincerepelling forces make dendrite tracking difficult in areas where thedendrite changes direction often.A176

Page 4/6 Version 1.1 04/20/05Microscope controlThe nanoManipulator (nM) application provides an intuitive interface to scanning-probe microscopes,allowing scientists from a variety of disciplines to examine and manipulate nanometer-scale structures.(Taylor II, Robinett et al. 1993)The nM displays a 3D rendering of the data as it arrives in real time. Usinghaptic feedback controls, the scientist can feel the surface representation to enhance understanding ofsurface properties and can modify the surface directly. Studies have shown that the nM greatly increasesproductivity by acting as a translator between the scientist and the instrument being controlled. (Finch, Chiet al. 1995)The haptic feedback component of our system has always been exciting to the scientists on the team;they love being able to feel the surface they are investigating. However, it is during modification that hapticfeedback has proved itself most useful, allowing finer control and enabling whole new types ofexperiments. We describe here particular benefits received by adding haptic feedback to this application.Haptic feedback has proved essential to finding the right spot to start a modification, finding the path alongwhich to modify, and providing a finer touch than permitted by the standard scan-modify-scan experimentcycle. (Taylor II, Chen et al. 1997)Finding the right spotDue to time constants and hysteresis in the piezoceramic positionersused by SPMs to move the tip, the actual tip position depends on pastbehavior. The location of the tip for a given control signal is different if itis scanned to a certain point than if it is moved there and left constant.PositionThis makes it difficult to plan modifications accurately based only on anwhenimage made from scanned data.scanningPositionHaptic feedback allows the user to locate objects and features on the after being heldsurface by feel while the tip is being held still near the starting point for still for several secondsmodification. Surface features marking a desired region can be locatedwithout relying only on visual feedback from the previous scan. This allowed a collaborator to position theA177

Page 5/6 Version 1.1 04/20/05tip directly over an adenovirus particle, then increase the force to cause the particle to dimple directly in thecenter. It also allowed the tip to be placed between two touching carbon filaments in order to tease themapart.Finding the right pathEven given perfect positioners, thescanned image shows only the surface as itwas before a modification began. There isonly one tip on an SPM: it can either bescanning the surface or modifying it, butnot both at the same time. Haptic feedbackduring modification allows one to guidechanges along a desired path.This sequence shows haptic feedbackbeing used to maneuver a gold colloidparticle across a mica surface into a gapthat has been etched into a gold wire. Theyellow lines indicate where the userpushed with high force. This gap forms atest fixture to study the energy states of theball. The colloid is fragile enough that itwould be destroyed by getting the tip ontop of it with modification force or by many pushes. This prevents attempts to move it by repeatedprogrammed “kicks”. Haptic feedback allowed the user to tune the modification parameters so that the tipbarely rode up the side of the ball while pushing it. This allowed the guidance of the ball during pushing sothat only about a dozen pushes were required.Haptic feedback was also used to form a thin ring in a gold film. A circle was scraped to form theinside of the ring, leaving two “snow plow” ridges to either side. By feeling when the tip bumped upagainst the outside of the outer ridge, another slightly larger circle was formed. This formed a thin gold ringon the surface.A light touch: observation modifies the systemWhen deposited on the surface, carbonnanotubes are held in place by residue fromthe solution in which they are dispersed. Onsome surfaces, the tubes slide freely oncedetached from the residue until they contactanother patch of residue or another tube.Even the light touch of scanning causes themto move. By using only touch mode andswitching between imaging and modificationforce, we have been able to move and reorientone carbon tube across a surface andinto position alongside another tube. Once settled against the othertube, it was stable again and we could resume scanning to image thesurface. Haptic feedback and slow, precise hand motion (“haptic imaging”) allowed us to find the tube atintermediate points when we could not scan. The fact that the surface cannot be imaged at intermediatestages prevents this type of experiment from being performed using the standard scan-modify-scan cycle.Haptic visualization moving forwardThe routine application of haptic feedback has reached the stage where computer graphics was yearsago: Phong shading and some basic texturing operations. Building on techniques suggested by Minsky,Fritz and others, it is time for applications to push further. They should make use of multiple surfacecharacteristics to carry information about multiple data sets. Some options being investigated are:• Compliance (stiffness) of the simulated surfaceA178

Page 6/6 Version 1.1 04/20/05• Friction models (coulomb, viscous and drag)• Adhesion• Texture (image-based or procedural, stationary or probabilistic)• Surface vibrationAgain, the true potential for haptic feedback seems to lie in its unique position as the only bidirectionalmodality. It allows both feeling and modifying at the same time. This allows direct andimmediate control over simulations and remote operations with direct and immediate sensing of the results.At UNC, investigation into the appropriate linear mapping for each of the these forces is being led byMark Hollins. Early work in this area is described in (Seeger, Henderson et al. 2000), with the perceptualstudies underlying this use described more fully in (Hollins, Seeger et al. 2004). It turns out that theinterplay between dimensions (friction, stiffness, etc) is non-trivial, and care must be taken when more thanone is presented at a time. (Hollins, Seeger et al. 2004)ReferencesASEL (1998). http://www.asel.udel.edu/sem/research/haptics/.Avila, R. S. and L. M. Sobierajski (1996). A Haptic Interaction Method for Volume Visualization.Proceedings of IEEE Visualization '96, San Francisco. 197-204.Brooks, F. P., Jr., M. Ouh-Young, et al. (1990). Project GROPE - Haptic displays for scientificvisualization. Computer Graphics: Proceedings of SIGGRAPH '90, Dallas, Texas. 177-185.DiFranco, D. E., G. L. Beauregard, et al. (1997). The effect of auditory cues on the haptic perception ofstiffness in virtual environments. Proceedings of the ASME Dynamic Systems and ControlDivision, ASME. 17-22.Finch, M., V. Chi, et al. (1995). Surface Modification Tools in a Virtual Environment Interface to aScanning Probe Microscope. Computer Graphics: Proceedings of the ACM Symposium onInteractive 3D Graphics, Monterey, CA, ACM SIGGRAPH. 13-18.Fritz, J. and K. Barner (1996). Stochastic models for haptic texture. Proceedings of the SPIE InternationalSymposium on Intelligent Systems and Advanced Manufacturing - Telemanipulator andTelepresence Technologies III, Boston, MA.Fritz, J. P. (1996). Haptic Rendering Techniques for Scientific Visualization. Electrical Engineering,University of Delaware: 148.Hollins, M., A. Seeger, et al. (2004). "Haptic perception of virtual surfaces: Scaling subjective qualities andinterstimulus differences." Perception 33: 1001-1019.Iwata, H. and H. Noma (1993). Volume Haptization. Proc. IEEE 1993 Symposium on Research Frontiers inVirtual Reality. 16-23.Minsky, M. (1995). Computational Haptics: The Sandpaper System for Synthesizing Texture for aForFeedback Display. Program in Media Arts and Sciences, MIT: 217.Minsky, M., M. Ouh-young, et al. (1990). Feeling and Seeing: Issues in Force Display. Proceedings ACMSiggraph Symposium on Interactive 3D Graphics.Mitsuishi, M., Y. Hatamura, et al. (1993). Auditory and Force Display of Key Physical Information inMachining/Handling for Macro/Micro Teleoperation. Proceedings of the 1993 IEEE InternationalConference on Robotics and Automation, Atlanta, Georgia. 137-169.Ouh-young, M. (1990). Force Display In Molecular Docking. Computer Science. Chapel Hill, University ofNorth Carolina: Tech Report #90-004.Seeger, A., A. Henderson, et al. (2000). Haptic Display of Multiple Scalar Fields on a Surface. Workshopon New Paradigms in Information Visualization and Manipulation, ACM Press.Srinivasan, M. A., G. L. Beauregard, et al. (1996). The impact of visual information on the hapticperception of stiffness in virtual environments. Proceedings of the ASME Dynamics Systems andControl Division, ASME. 555-559.Taylor II, R. M., J. Chen, et al. (1997). Pearls Found on the way to the Ideal Interface for Scanned-probeMicroscopes. Visualization '97, Phoenix, AZ, IEEE Computer Society Press. 467-470.Taylor II, R. M., W. Robinett, et al. (1993). The Nanomanipulator: A Virtual-Reality Interface for aScanning Tunneling Microscope. SIGGRAPH 93, Anaheim, California, ACM SIGGRAPH. 127-134.A179

DAB: Interactive Haptic Painting with 3D Virtual BrushesBill Baxter Vincent Scheib Ming C. Lin Dinesh ManochaDepartment of Computer ScienceUniversity of North Carolina at Chapel Hillbaxter,scheib,lin,dm@cs.unc.eduhttp://www.cs.unc.edu/geom/DABAbstract: We present a novel painting system with an intuitivehaptic interface, which serves as an expressive vehicle for interactivelycreating painterly works. We introduce a deformable,3D brush model, which gives the user natural control of complexbrush strokes. The force feedback enhances the sense of realismand provides tactile cues that enable the user to better manipulatethe paint brush. We have also developed a bidirectional, two-layerpaint model that, combined with a palette interface, enables easyloading of complex blends onto our 3D virtual brushes to generateinteresting paint effects on the canvas. The resulting system, DAB,provides the user with an artistic setting, which is conceptuallyequivalent to a real-world painting environment. Several usershave tested DAB and were able to start creating original art workwithin minutes.Keywords: Haptics, Human Computer Interaction, Painting Systems,Deformable Brush Model1 IntroductionThe art of painting refers to the aesthetic aspects of a painterlywork. The craft of painting deals with the study of materials, includingpaint medium, tools, supports, and methods, i.e. the manipulationof materials to express an artist’s intent and purpose[May70]. The art and craft of painting are closely related: an artistcannot divorce one from the other. Nevertheless, recent technologicaladvances in computer graphics have largely centered around theart of painting, with little attention being given to the craft.Commercial painting systems and recent research on the generationof painterly works have mainly emphasized the appearanceof the final product. However, the word ‘painterly’ also describesa fusion of feeling and action, sight and touch, purpose and paint,beyond merely producing an image that gives an artistic impression[May70].Rather than focus primarily on the rendered appearance, theremay be equal merit in recreating the “sight, touch, action and feeling”of the artistic process itself. By designing a setting for artiststo freely and creatively express themselves, as they would in a traditionalpainting environment, computer graphics can serve as aconduit to the craft as well.1.1 Main ContributionOur primary goal is to provide an expressive vehicle for interactivelycreating original painterly works with computer systems.We present a physically-based, deformable 3D brush model, whichgives the user control of complex brush strokes intuitively. The hapticfeedback enhances the sense of realism and provides tactile cuesthat enable the user to better manipulate the paint brush. We haveFigure 1: An original work created using DAB. (Rebecca Holmberg,artist)also developed a bidirectional, two-layer paint model that, in combinationwith a palette interface, enables easy loading of complexblends onto our 3D brush model and generates interesting paint effectson the canvas.We have attempted to provide a minimalistic interface that requiresas few arcane buttons, key-presses, and complex controls aspossible, yet still offers a great deal of expressive power. With ourhaptic painting system, DAB, most paintings can be created withjust the force-feedback device and the space bar on the keyboard.In comparison to the existing computer painting programs, our approachoffers the following advantages:¯ Natural and expressive mechanisms for manipulating thepainting tools, including brushes, palette, paint and canvas;¯ Simple and easy loading of complex blends using 3D virtualbrushes;¯ Physically-based and realistic brush footprints generated automaticallyby the brush strokes;¯ Intuitive and familiar feel of the painting process requiringlittle or no training.Our haptic painting system, DAB, has been tested by a numberof users. A novice user can start painting with just a few (typicallyless than ten) minutes of simple instruction. Fig. 1 shows a paintingcreated by an amateur artist with DAB. Since DAB providesa familiar setting, conceptually equivalent to a real-world paintingA180

environment, an artist need only control the virtual brush as he orshe would a real brush. This interface could also be combined withmost of the existing interactive painting programs or used as aneffective training system for painting.1.2 Prior WorkComputer-Generated Painting: A number of researchers havedeveloped automatic methods for transforming ordinary imagesinto painterly or otherwise imaginative renderings [Her98, Lit97,Mei96]. Others have developed 2D methods for simulating thelook of painting, from Alvy Ray Smith’s original “Paint” program[Smi78] to more recent physically-based approaches [CPE92,CAS·97]. Commercial packages such as COREL’s Painter[COR00] are able to achieve realistic looking simulations of naturalmedia by clever use of 2D textures and compositing tricks. Theamount of training required to proficiently use these commercialpainting systems is large, as is the complexity involved in obtainingthe precise strokes desired, even for skilled painters.Modeling of Brushes: Several researchers have endeavored to accuratelymodel the appearance of real brush strokes, but most techniqueshave been 2D heuristics. Strassmann modeled a brush as aone-dimensional array of bristles swept over a trajectory defined bya cubic spline curve [Str86]. This work was able to account for anumber of effects achievable with an actual brush, such as varyingcolor, width, and wetness. Wong and Ip [WI00] defined a complexset of interrelated parameters to vary the density, opacity, andshape of a footprint in a way that takes into account the behavior ofa three-dimensional round calligraphy brush. The resulting strokeappearances are informed by the physical behavior of the brush, butare not actually physically generated. The method as described isonly partially interactive.Our approach for brush modeling shares some similar themeswith the work of Saito [SN99] on modeling a physical 3D brush forJapanese calligraphy and sumie paintings. However, our techniqueis more flexible in terms of brush shape, dynamics, and loading,and is able to take advantage of 3D graphics hardware as well.User Interface: Hanrahan et al. allowed the user to paint directlyonto a 3D model by using standard graphics hardware to map thebrush from screen space onto the model [HH90]. Commercial systems,such as Z-Brush [Pix00] and Deep Paint [hem00], also allowusers to paint directly on surfaces, but this is accomplished withstandard 2D brush footprints that are projected onto the surface ofthe 3D object. The brush itself is not three-dimensional.Several of the more advanced commercial tools, e.g. Painter,support pen-based input with sophisticated 5-DOF tablet devices,yet most still use only the position and pressure parameters andignore the tilt. Further discussion on tablet systems is given in Section6.The idea of 3D painting has been explored in [ABL95, JTK·99,GEL00] using a simple, rigid 3D brush (tip) controlled by a 6-DOFinput device to color 3D surfaces. All these 3D painting systemswere restricted to monochrome brushes.1.3 OrganizationThe rest of the paper is organized as follows. Section 2 gives anoverview of our approach and the user interface. We present themodeling of the paint brushes in Section 3 and force display usinghaptic devices in Section 4. Section 5 describes our techniquesfor rendering acrylic or oil-like paint. Next, we briefly highlightthe implementation of our prototype painting system with a hapticinterface and demonstrate its features via the actual paintings ofseveral volunteers in Section 6.Figure 2: System Architecture2 ApproachIn this section we give an overview of our approach and the userinterface of our haptic painting system.2.1 OverviewWe have developed a novel, physically-based, deformable 3D brushmodel integrated with a haptic interface. The haptic stylus servesas a physical metaphor for the virtual paint brush. It takes in theposition and orientation of the brush and displays the contact forcebetween the brush and the canvas to the user. The bristles of thebrush are modeled with a spring-mass particle system skeleton anda subdivision surface. The brush deforms as expected upon collidingwith the canvas. This framework allows for a wide selection ofbrush types to be made available to artists.Our multi-layered paint model supports important features ofpaint, such as bidirectional paint transfer, blending, drying, andcomplex brush loading. The surfaces of the brush, canvas, andpalette are coated with paint using this model. A schematic diagramis shown in Fig. 2 to illustrate how various system components areintegrated.2.2 User InterfaceWe use a SensAble Technologies’ PHANToM as a haptic deviceand a dual-processor Pentium III PC with NVIDIA’s GeForce2graphics card. One processor is dedicated to force display and theother is used to compute the brush dynamics and the paint transferand blending. Fig. 3 shows the physical setup of our system.Figure 3: Haptic Painting System Setup: An artist using a hapticstylus to paint directly on the virtual canvas using DAB.Our haptic painting system allows the user to paint directly ontoa virtual canvas displayed on the screen. Using the space bar as atoggle, the user can bring up the virtual palette for paint mixingand brush cleaning, or put the palette aside to paint directly ontothe canvas. The user is also presented with a wide selection ofvirtual brushes that mimic different types and shapes of brushesused in traditional painting. A simple menu is presented for savingand loading a clean or previously painted canvas, undoing a brushstroke, quickly drying the canvas partially or completely, etc. Fig. 4shows a snapshot of our graphical user interface, consisting of thevirtual brushes, the palette, and the canvas.The paint brush deforms in a natural, physical way, as the usermoves the brush across the canvas. The user can create strokes withthe brush, which behaves much in the way a real brush would. Theactual footprints of the brush and resulting strokes are generatedbased on the user’s manipulation of the 3D brush on the virtualcanvas.A181

Table 1: We show some real brushes, our model for each (skeletal structure and surface mesh), and example strokes generated with each.Figure 4: Graphical User Interface: The virtual canvas with thebrush rack and a portion of the palette (LEFT); the brush rack andthe palette for color mixing (RIGHT).3 Modeling of 3D BrushesPaint brushes are often regarded as the most important tools at anartist’s disposal. A good set of brushes can enable a competentartist to create virtually any effect he or she can imagine, from theintricate detail of cresting waves, wispy billowing clouds, to thesubtly blended shifting hues in a sunset. In this section, we describeour techniques for modeling 3D virtual brushes.3.1 Introduction to BrushesFigure 5: Basic Brush AnatomyFig. 5 shows the anatomy of a typical brush. Brush heads aremade with a variety of bristles, natural soft animal hair, and syntheticmaterials. Some of the most common styles for brushes usedin oil-like painting [May70] are:¯ Rounds. Have a simple tubular shape with a semi-blunt point,allowing for a great variety of strokes.¯ Flats. Thinner and wider than rounds with bristles squaredoff at the point. Flats are typically longer than they are wide.¯ Brights. The same shape and construction as flats but typicallyshorter, with width nearly equal to length.¯ Filberts. Have a thicker collection of bristles that increaseability to hold paint. Filberts usually have oval-shaped heads.There are other types of specialty brushes, such as fans andblenders, but the four above are the most versatile and widely used.The second column of Table 1 shows images of each type.3.2 Overview of Modeling ApproachTo model a 3D paint brush requires developing both a geometricrepresentation and a physics-based model for its dynamic behavior.The requirements of an interactive haptic painting system placeconstraints on the design: the brush dynamics must run at interactiverates and remain stable under all types of user manipulation.We model the brush head as a subdivision surface meshwrapped around a spring-mass particle system skeleton. The particlesystem reproduces the basic motion and behavior of a brushhead, while the deformable mesh skinned around this skeleton representsthe actual shape of the head. We also derive an approximatedimplicit integration method based on an existing numericaltechnique for cloth simulation [DSB99] to take large integrationsteps while maintaining stability. Although our brush model mayappear simplistic at first, it is designed to capture the essential qualityof physical brushes to maintain interactivity at minimal computationalcosts.Based on our generalized 3D brush model, we are able to adjustsome key parameters to generate the different types and shapes ofbrushes and mimic their physical behavior. In Table 1, we show thegeometric structure used to construct each of the brushes describedin Section 3.1. We also show the deformation of different brushesas they make contact with the canvas.3.3 Brush DynamicsThe difficulty in simulating the paint brushes used in acrylic andoil-like painting is that the brushes are numerically stiff dynamicalsystems, and suffer from numerical instability. Bristles havevery little mass. As they bend, energy stored in them can inducelarge accelerations and velocities when they are abruptly released.The brushes also behave as highly damped systems and we use thisproperty to improve the stability of our solver.We have considered and evaluated several numerical methodsfor particle system simulation, but we found the approximated implicitintegrator presented in [DSB99] to be most effective for thisapplication, primarily because of its stability. We simulate somebrushes with the approximate implicit integrator and others with avariation based on first-order dynamics.3.3.1 Newtonian DynamicsThe motion of the particle system representing the brush can bedescribed mathematically by Newton’s second law, a second orderdifferential equation, decomposed here as a pair of coupled first-A182

order differential equations: Ü ÚÚÅ ½ In this equation, x is a ¿Ò vector containing the spatial coordinatesof Ò particles, Ú is a ¿Ò vector of particle velocities, and f is a ¿Òvector of the forces on those particles. M is a ¿Ò¢¿Ò diagonal matrixwhose diagonal entries are of the form Å Ñ ¿ , whereÑ is the mass of particle .The semi-implicit method for simulation of deformable objects[DSB99] approximates a solution to the equations of motion inthree stages:1. Implicit integration of linear force components2. Approximate post-correction to account for non-linear forcecomponents3. Deformation constraint enforcement to prevent excessivestretchThe resulting solution is much less accurate than other largestepintegration techniques, such as that presented by Baraff andWitkin [BW98], but it is both more stable and computationally lessdemanding. The speed advantage comes from separating out thelinear force component, which allows one to solve the equations ofmotion using just a matrix-vector multiply each step. The integrationstep has the form: ¡Ü¡ÚÁ´Ú ¼ ·¡Úµ ÐÜ ½Å½ Ð´Ü ¼µSince only the linear force components are handled by the integrationstep, ´Á Рܵ ½ becomes a constant matrix.This method works well for cloth, which generally has weakbending forces, but for brush simulation, approximating the effectof the non-linear force components leads to local errors in angularmomentum. We have observed that with some brush skeletons, thesolver effectively ignores stiff bend forces, leading to brushes withtoo much angular motion. Achieving stiff brush behavior is possible,but depends upon the skeletal structure. Section 3.5 discussesour brush construction in more detail.The final step in the method is the application of deformationconstraints. In this step, particles are assumed to have traveled inthe right direction, but perhaps too far, inducing excessive stretchin the material. This is corrected by simply altering the positionsof particles, iteratively contracting the springs in the material untiloverstretch is eliminated. The deformation constraints play a majorrole in the overall stability of the system by ensuring that at the endof every step, every spring is in a physically plausible configuration.For collision handling and contact response, particles found tobe penetrating the canvas are projected up to the nearest point onthe surface. We also add a frictional drag force to colliding particlesfor more realistic results. We model the frictional drag as friction normal Ú tangentialwhere is the coefficient of friction.3.3.2 Aristotelian DynamicsReal bristles empirically obey the Aristotelian model of physics,which is characterized by the lack of inertia. In this model, objectsmove only for the duration that forces are applied. Inspiredby [WB97], we use this simplified model to simulate most of ourbrushes. This has advantages for speed, stability, and in some casesusability. With the Aristotelian dynamics model, the motion ofthe particle system is represented by a single first-order differentialequation: Ü Å ½ Since objects now stop moving instantly in the absence offorces, the result is motion that appears heavily damped, which(1)(2)is precisely how we wish brushes to behave. In the second ordermodel, to simulate this damping requires adding large dampingforces to the system to cancel out the large velocities induced bystiff springs. Using a first-order physics model, however, we cancircumvent this step entirely.We modify the approximated implicit integration formula asfollows for the first order model:¡Ü Á ÐÜ ½Å ½ Ð´Ü ¼µSince this equation is still in the same form as Eqn. 2, most of theintegration technique remains unchanged. An exception is that weomit the frictional damping force during collisions and just modifyvelocity, since in the first order model the two have the same effect.3.4 Brush SurfaceWe use subdivision surfaces as the geometric representation for thebrush head because of their ability to represent arbitrary topologyand vertices of arbitrary valence easily. The brush head subdivisionsurface is defined by control points anchored relative to the massparticles. It is possible to use either interpolating or approximatingsubdivision schemes for the brush surface.An interpolating scheme eases the task of choosing reasonablecontrol vertices for the rough mesh, since the limit surface is guaranteedto pass through each of them. In fact, since all vertices at allsubdivision levels are on the limit surface, it also facilitates changingthe tessellation level of the mesh. However, due to frequentappearance of high curvature in the resulting surface, often interpolatingsurfaces do not deform as smoothly as would be expected ofa brush.Approximating schemes generate surfaces that are generallysmoother and fairer, but it is more difficult to place control pointsto achieve the desired surface. The extensions to the Loop approximatingscheme presented by [HDD·94] would be useful for accuratelymodeling sharp features like the finely tapered point of around brush.In our implementation we chose to use a triangular base meshand to subdivide with the interpolating Butterfly rule to make thetask of generating the brush control mesh easier. Some exampleresults can be seen in Table 1.3.5 Brush GenerationGiven these dynamical models for simulation, we synthesize a fullset of brushes suitable for creating a wide variety of paintings. Onetype of brush is modeled as a single spine composed of a linearchain of Ò particles. With our integration method and this structure,we are able to model the softer style of brushes used in Japanesecalligraphy, called fude. Our fude brushes work best with the firstorder dynamics model, which makes the brush appear more solidby eliminating oscillations.We model stiffer brushes, like those used in oil and acrylicpainting, by using a more complicated skeletal structure. The basicbuilding block for our stiff brushes is five mass particles connectedwith springs to form a pyramidal truss. The round brush consists ofone of these trusses. The four particles that form the base are rigidlyfixed to the brush handle and are directly driven by the user’s input.The fifth particle serves as the point of the brush.Table 1 shows a summary of the brush models and gives examplesof the strokes that can be generated with each. Wide brushesare formed from two trusses, and filberts are generated from four ofthem, the outer two being shorter than the inner two. We use eachbrush structure to define an entire family of brushes of differentsizes by parametrically varying the scaling along the three cardinalaxes.4 Haptic DisplayAn important aspect of our 3D painting system is the ability to providesufficiently good force feedback to emulate the sensation ofapplying brush strokes to a canvas. Our 6-DOF armature inputA183

device also serves as a 3-DOF force output device. We align thevirtual paintbrush with the physical 6-DOF stylus, and position itso that the point of 3-DOF force delivery coincides with the pointwhere the head meets the handle on the virtual brush. In this section,we present our approach for force display.4.1 Decoupled HapticsWe separate the force computation from the brush deformationcomputation, since the two have different goals. For instance, thenon-dynamical deformation constraints used by the approximatedimplicit solver are acceptable for approximating the visual aspectsof brush behavior, but are not appropriate for force simulation. Furthermore,the force updates for haptic feedback need to be generatedat close to 1kHz for smooth jitter-free output, but the deformationcalculation only needs to proceed at visual update rates (around30Hz). Consequently we decouple the force simulation from brushdynamics simulation, and simplify the force computation to run atkHz rates.4.2 Basic Force ModelThe root of our force model is a simple piecewise linear functionof the penetration depth of the undeformed brush point. If Ô is thepenetration depth , and Ð Ô is the length of the brush head projectedonto the canvas normal, Ò, then the force is modeled as:´¼ if Ô ¼ ´ Ôµ Ò´ ½Ð Ôµ Ôif ¼ Ô Ð Ô (3)Ò´ ½ ·´ ¾Ð Ôµ´ Ô Ð Ôµµ if Ð Ô Ôwhere ½ is a small positive constant that models the light spring ofbristles and ¾ is a larger positive constant that simulates collisionof the actual brush handle with the canvas. The spring constantsare normalized by Ð Ô so that the same absolute force is deliveredwhen the handle first hits the canvas, regardless of the brush lengthor orientation. The value of ½ can be changed to simulate brushesof varying stiffness.4.3 Compressive EffectsWhen a real brush contacts the canvas at close to a right angle, thestiff bristles initially act as strong compressive springs, transmittingan abrupt force to the handle. As more pressure is applied, thebristles buckle and the compressive force reduces as bending forcestake over. When the brush makes a contact at an oblique angle,compressive effects play a lesser role in the force felt.Therefore, we extend the piecewise linear function, Eqn. 3, toa piecewise Hermite curve. This curve is defined by a series ofcontrol tuples which contain the penetration depth and correspondingforce magnitude, and the linear stiffness of the spring model atthat point. We currently use a four-segment piecewise curve, whichwas derived from the empirical observation of how a brush headbehaves under compression.The initial segment of the piecewise curve models the compressiveforce. We assign the initial control tuple a fairly strong linearspring constant to simulate the initial strong compressive force. Wemodulate this compressive force according to the angle of contact,by multiplying the force value of the second control tuple by anangle-dependent coefficient between one and zero. Given , theangle between the canvas normal and negated bristle direction vector,the factor we use is­ Ó× ¾´¾µ¼if otherwiseThis results in a compressive force that is strongest when a brushcontacts the canvas at a right angle and that tapers off to zero as thebrush approaches a 45 degree angle to the canvas.4.4 Frictional ForcesThe final component of the force delivered to the user is a smallamount of tangential resistance. Though small in magnitude, frictionalforces have a large effect on the user’s perceived ability to(4)control the brush by damping small oscillations in the user’s hand.We model friction Ø simply, as a force opposite the current brushvelocity, Ú , which is added to the other feedback forces: Ø Ø ´Ú Ò´Ò ¡ Ú µµwhere Ø is the coefficient of friction.5 Paint ModelComplementing our expressive brushes and force feedback, wepresent a paint model capable of capturing complex effects interactively.Our paint model incorporates variable wetness & opacity,conservation of volume, and a hardware-accelerated bi-directionalpaint transfer algorithm. It supports the following operations andtechniques expected from acrylic or oil painting, while maintainingcomplete interactivity.¯ Blending – Mixing of multiple pigments to obtain the desiredcolor.¯ Bi-directional transfer – Transferring paint both from thebrush to canvas, and back from the canvas to the brush.¯ Complex brush loading – Filling the various portions of thebrush head with different pigments.¯ Variable dryness – Controlling the blending of new paintonto previous layers by allowing paint to partially dry.¯ Glazing – Painting with translucent layers (veils) of colorsover other opaque colors (i.e. underpainting).¯ Impasto – Painting with thick volumes of paint without additionof any medium.Users can also generate similar results using other advanced paintingprograms. However, with our paint model, they need only manipulatethe virtual brushes similar to real ones, in order to automaticallygenerate the intended paint effects.5.1 Bi-directional Paint TransferPaint information is stored on both the canvas and brush in multipletextures (described in Section 5.2). The brush subdivision surface istessellated to a polygonal surface. When this surface intersects thecanvas geometry, the brush is considered to be in contact with thecanvas. The bi-directional transfer must correctly modify the painttextures to simulate paint volume being interchanged between thetwo surfaces. Figure 6 displays a brush stroke possible only withbi-directional paint transfer.Figure 6: Bi-directional paint transfer is demonstrated by dragginga yellow paint stroke through wet purple paint (LEFT). A purpleglaze of paint has been thinly applied over dry paint (RIGHT).The paint transfer problem is first reduced to two dimensionsto simplify computation while introducing only slight inaccuracies.In the general case, a projection plane would be chosen thatmaximizes the area projected by the intersecting curve between thebrush and canvas surfaces. Currently we have implemented only atwo dimensional canvas, and therefore use the canvas plane for theorthographic projection of the brush. This is achieved with polygonA184

asterization hardware, for speed and ease of implementation. Theprojected textures of the brush are used as the brush footprint.The textures must be updated to simulate paint transfer andmixing. This 2D blending of the footprint with the canvas is discussedin Section 5.3. The simulation of the brush produces discreteinstances of the brush surface; to produce a continuous strokethe blending operation is performed over a line connecting the currentfootprint to the previous one. The centroids of the footprintpolygons are used as endpoints. This method provides smoothstrokes while the footprint is not changing dramatically.After 2D blending is complete, the updated textures are reappliedto the surfaces. This is achieved by rendering a variation ofthe brush subdivision surface mesh. The surface vertices that wereprojected to the footprint are used as texture coordinates into thenow updated footprint textures. The original surface texture coordinatesare used as vertex locations to render back into the surface’stexture maps.5.2 Paint RepresentationThe 3D brush and transfer methods presented here can be combinedwith many media types such as paint, ink, or watercolor. The DABsystem currently includes a model that approximates the acrylic andoil families of paint.Each paint surface contains two color layers. These are referredto as the ‘surface’ and ‘deep’ layers. Conceptually, the surfaceis covered by only a thin surface layer, and more thoroughly bythe underlying deep layer. The surface layer is the boundary atwhich paint transfer between objects occurs. Surface layers arecompletely wet. The canvas’s deep layer represents the paint thatis completely dry. The brush’s deep layer represents the reservoirof paint contained within the bristles. The paint transfer betweensurface layers occurs upon a collision between two objects (i.e. thebrush and canvas). Transfer from the brush’s reservoir layer to thesurface is performed whenever the surface layer is no longer saturated(and paint remains in the reservoir layer). Drying paint fromthe canvas’s surface layer to the dry layer occurs on a timed intervalor as requested by the user.The surface and deep layers are stored in color textures. A representationof the volume of paint in each layer is stored in an attributetexture. The surface layers and brush reservoir layer usefixed point representations, while the dry layer of the canvas is aspecialized relative height field, and is described in Section 5.5.5.3 Paint MixingThe amount of volume transferred between surface layers is dependenton the volume of paint within each layer. The volume leaving,Î Ð , is computed from the initial volume, Î , and transfer rate, Ê,over the elapsed time, Ì , by the equation, Î Ð Î ¡ Ì ¡ Ê. Theresulting paint color, ÒÛ, is computed by the weighted portionsof remaining volume and color, Î Ö Î Î Ð and , and incomingvolume and color from the other surface, ÎÐ ¼ and :¼ ÒÛ Î Ö ¡ · Î ¼Ð ¡ ¼ This essentially additive compositing formula is easy to workwith, and gives predictable results, but does not model the way inwhich colloidal suspensions of pigment actually mix. The Kubelka-Munk model is the best known model available for accurately compositingpigments, but comes with significantly higher computationalcost. See for example [CAS·97].5.4 Optical CompositionTo generate realistic paint effects, the wet and dry layers of the canvasare composited together with an embossing of the paint volume.This allows for glazing effects, as illustrated in Fig. 6. The volumeof the wet layer, Î Û, is multiplied by the optical thickness, Ç Ø,ofthe paint, and then used for alpha blending the wet and dry layercolors, Û and . ×ÔÐÝ « ¡ Û ·´½ «µ ¡ « ÑÒ´Î Û ¡ Ç Ø ½µ5.5 Drying the CanvasOur paint model also supports variable wetness as shown in Fig. 7.Variable wetness is accomplished by gradually moving paint fromthe completely wet surface layer of the canvas to the completelydry deep layer.Figure 7: Variable wetness is displayed as yellow paint has beenpainted over the purple color stripes of 100%, 50%, 0%, 75%, 25%dryness (left to right).The composited color of the paint must not change during drying.The optical blending function is used with this constraint tosolve for the new dry layer color, , ¼ when some volume, Æ«, isremoved from the wet layer. ¼ «¡Û·´½ «µ¡ « ¼ Û « ¼ « Æ«´½ « ¼ µThe dry layer of the canvas uses a relative height field to allow forunlimited volume of paint to be added, with a constraint only onthe relative change in height between texels. An embossing of theheight field is also computed. We use additive blending to combinethis embossing and the color buffer to create the final renderedimage of the paint.6 Implementation ResultsWe have developed a prototype painting system, DAB, which combines3D virtual brushes with a haptic interface and our paintmodel, as described in this paper. As mentioned in Section 2, thegraphical user interface consists of three main elements: the canvas,the palette and the brush rack.In the absence of a 3D stereo display, we have introduced shadowsin our graphical display to enable the users to infer the relativeposition of the paint brush to the virtual canvas.6.1 DiscussionA painter’s palette not only “lists” available colors, but also allowsa painter to mix and create a nearly unlimited number of new ones,and it presents both possibilities simultaneously through a simple,unified interface. Furthermore, creating complex color “gradients”on a painter’s palette is just as easy as creating a single color: simplymix the constituent colors less thoroughly. In short, a realpalette is a natural interface for color choosing, but one which hasnot been taken advantage of in previous computer painting systems.To take best advantage of a painter’s palette interface requiresa 3D virtual brush like the one presented in this paper. With a 3Dvirtual brush, loading the complex blends created on the palette issimple, as is creating strokes that use those blends. Combined withan appropriate 3D input device, DAB offers a powerful yet simpleinterface for painting.We chose to use a Desktop PHANToM for input and hapticfeedback because it provides true 6-DOF input with excellent precisionand low noise, while offering fully programmable 3DOF forceoutput. Other input devices such as tablets offer at most 5-DOFinput (lacking a degree of freedom for twist), and have rather largenoise in the tilt measurements.A185

On a pragmatic level, the force feedback is useful in that it enablesthe user to detect and maintain contact with the canvas betterthan if just shadow cues are provided. A tablet gives a physical surfacethat serves the same purpose, but it always gives the sensationof a rigid pen on a hard surface, rather than a soft, flexible brushon a canvas. Nearly all the users who have used both a tablet systemand a haptic device preferred the soft feel of force feedback forbrush simulation. Finally, with fully programmable forces, we arealso able to change the feel of the brush at will, making it softer orharder for instance.We are currently planning a detailed user study to thoroughlyevaluate and assess the value of force feedback in creating the “rightfeel” for the artists. Using a programmable force feedback devicewith a true 3D workspace further enables the possibility to expandour system in a number of exciting directions covered in the nextsection.6.2 User FeedbackMore than fifteen users have painted with our system. This group ofusers includes amateurs and art students, both males and females,with ages ranging mostly from early 20’s to late 30’s. Some haveprior experience with other computer painting programs and varioususer interfaces. All the users were able to pick up the hapticstylus and start painting immediately, with little training or detailedinstruction. A small selection of their artistic creations is shown inFigs. 8 to 14. Additional images of art works created by our users,and detailed screen shots, are available as supplemental materialson the CD-ROM and on the project website.Among users who have worked with other painting programsand interfaces, most found our painting system to be more intuitive.For artists with prior painting experience, our painting system wassubstantially easier to adapt to than other painting programs, whileoffering similar capabilities, such as undoing brush strokes, dryingpaint, etc. We attribute this to the fact that DAB offers a paintingenvironment that takes advantages of skill transfer. DAB also seemsto have an appeal for people with an artistic bent, but who wouldnot normally consider painting, as well as for painters who wouldnot normally use a computer painting system.7 Future WorkUsers of all types found DAB compelling to work with, howeverthere are many aspects of the system which can be extended.For improved accuracy in the brush deformation simulation, wecontinue to investigate the use of other efficient integration and simulationmethods such as [BW98]. We are also interested in simulatinga greater range of haptic phenomena from the feel of painttextures, to the variation in sensation when using different types ofbrush fibers, from painting on different backings, or with differentmediums. Another natural step would be to go from painting 2Dsurfaces to painting 3D geometric models.Our current paint model can be extended to depict more advancedcharacteristics of oil painting such as: gouging effects frombristle marks, anisotropic BRDFs, multiple wet layers of paint, andlighting-independent rendering of paint bumps. We are also interestedin a real-time implementation of the Kubelka-Munk modelfor compositing. Expanding the set of virtual tools to include moretypes of brushes and and other artistic tools is also of interest.Our initial observations taken from a relatively small group ofamateur artists, art students, and novices indicate that our approachis effective. We plan to conduct a more thorough and extensive formaluser study over a larger group of users to confirm this observation,as well as to evaluate the effectiveness of various contributingfactors in our interface design.8 AcknowledgementsWe are grateful to all the people who evaluated our system. Severalof their paintings drawn using DAB are included in this paper.We would also like to thank the anonymous reviewers for theiruseful comments. This research was supported in part by ARODAAG55-98-1-0322, DOE ASCII Grant, NSF NSG-9876914, NSFDMI-9900157, NSF IIS-982167, ONR Young Investigator Award,and Intel.References[ABL95][BW98]M. Agrawala, A. Beers, and M. Levoy. 3D painting on scanned surfaces.In the Proc. of 1995 Symposium on Interactive 3D Graphics, pages 145–150. ACM SIGGRAPH, April 1995.D. Baraff and A. Witkin. Large steps in cloth simulation. Proc. of ACMSIGGRAPH, pages 43–54, 1998.[CAS·97] C. Curtis, S. Anderson, J. Seims, K. Fleischer, and D. Salesin. Computergeneratedwatercolor. Proc. of SIGGRAPH’97, pages 421–430, 1997.[COR00] COREL. Painter. http://newgraphics.corel.com/products/painter6.html,2000.[CPE92] T. Cockshott, J. Patterson, and D. England. Modelling the texture ofpaint. Computer Graphics Forum (Eurographics’92 Proc.), 11(3):C217–C226, 1992.[DSB99] M. Desbrun, P. Schroder, and A. Barr. Interactive animation of structureddeformable objects. Proc. of Graphics Interface’99, 1999.[GEL00] A. Gregory, S. Ehmann, and M. C. Lin. inTouch: Interactive multiresolutionmodeling and 3D painting with a haptic interface. Proc. of IEEEVR Conference, 2000.[HDD·94] H. Hoppe, T. DeRose, T. Duchamp, M. Halstead, H. Jin, J. McDonald,J. Schweitzer, and W. Stuetzle. Piecewise smooth surface reconstruction.In Proceedings of ACM SIGGRAPH, pages 295–302, 1994.[hem00] RIGHT hemisphere. Deep paint. http://www.us.deeppaint.com/, 2000.[Her98][HH90][JTK·99][Lit97]A. Hertzmann. Painterly rendering with curved brush strokes of multiplesizes. Proc. of ACM SIGGRAPH’98, pages 453–460, 1998.P. Hanrahan and P. Haeberli. Direct WYSIWYG painting and texturingon 3D shapes. In Computer Graphics (SIGGRAPH ’90 Proceedings),volume 24, pages 215–223, August 1990.D. Johnson, T. V. Thompson II, M. Kaplan, D. Nelson, and E. Cohen.Painting textures with a haptic interface. Proceedings of IEEE VirtualReality Conference, 1999.P. Litwinowicz. Processing images and video for an impressionist effect.Proc. of SIGGRAPH’97, pages 407–414, 1997.[May70] R. Mayer. The Artist’s Handbook of Materials and Techniques. TheViking Press, 1970.[Mei96]B. Meier. Painterly rendering for animation. In SIGGRAPH 96 ConferenceProceedings, pages 477–484. August 1996.[Pix00] Pixologic. Z-brush. http://pixologic.com, 2000.[Smi78] Alvy Ray Smith. Paint. TM 7, NYIT Computer Graphics Lab, July 1978.[SN99] S. Saito and M. Nakajima. 3D physically based brush model for painting.SIGGRAPH99 Conference Abstracts and Applications, page 226, 1999.[Str86] S. Strassmann. Hairy brushes. Computer Graphics (SIGGRAPH’86Proc.), 20:225–232, 1986.[WB97] A. Witkin and D. Baraff. Physically Based Modeling: Principles andPractice. ACM Press, 1997.[WI00] H. Wong and H. Ip. Virtual brush: A model-based synthesis of chinesecalligraphy. Computers & Graphics, 24, 2000.Figure 8: A painting by Eriko Baxter (LEFT); by Rebecca Holmberg(RIGHT)A186

Figure 9: A painting by Rebecca HolmbergFigure 12: A painting by Andrei StateFigure 10: A painting by Rebecca HolmbergFigure 13: A painting by Lauren AdamsFigure 11: A painting by Rebecca HolmbergFigure 14: A painting by Sarah HoffA187

ArtNova: Touch-Enabled 3D Model DesignMark Foskey Miguel A. Otaduy Ming C. LinUniversity of North Carolina at Chapel Hill{foskey,otaduy,lin}@cs.unc.eduhttp://www.cs.unc.edu/∼geom/ArtNovaAbstractWe present a system, ArtNova, for 3D model design witha haptic interface. ArtNova offers the novel capability ofinteractively applying textures onto 3D surfaces directly bybrush strokes, with the orientation of the texture determinedby the stroke. Building upon the framework of inTouch[GEL00], it further provides an intuitive physically-basedforce response when deforming a model. This system alsouses a user-centric viewing technique that seamlessly integratesthe haptic and visual presentation, by taking into accountthe user’s haptic manipulation in dynamically determiningthe new viewpoint locations. Our algorithm permitsautomatic placement of the user viewpoint to navigate aboutthe object. ArtNova has been tested by several users andthey were able to start modeling and painting with just afew minutes of training. Preliminary user feedback indicatespromising potential for 3D texture painting and modeling.Keywords: Haptics, Modeling, 3D Painting, Textures1 IntroductionDesigning 3D digital models is an important part of the productionprocess in VR, computer game design, entertainmentand education. Model design involves both 3D geometryand surface appearance, and each component offers itsown challenges. Designing 3D shapes is difficult when one’sinput tool has only two degrees of freedom; and painting asurface is further complicated by the fact that the screen isflat while the object being painted can be curved and nonconvex.With a physical model one could simply shape it inthree dimensions and then paint on its surface. We proposeto make model design easier by emulating that kind of experienceas faithfully as possible, while offering users moreflexibility and power via digital media.In physical sculpting, the sense of touch is essential.Force display is now making it possible to add some degreeof haptic feedback to the 3D sculpting experience. As anearly attempt to provide touch-enabled modeling features,a non-commercial plug-in to Alias|Wavefront’s Power Animatorsoftware package was developed at SensAble Technologies[Mas98], but force update rates were too low toprovide a realistic feel. Only recently have commercial hapticsculpting systems, such as F reeF orm TM [ST99], beenintroduced. They allow artists and designers to express theircreativity with 3D-T ouch TM .Figure 1. A turtle modeled and painted using Art-Nova. Notice the patterns on its back and legs.1.1 Main ResultsOur goal is to develop a digital model design sytem that supportsgeometric modeling and texture painting with a direct3D interface via force-feedback devices. Our system offers3D texture painting on arbitrary polygonal meshes withthe haptic stylus as an “electronic paintbrush”. Unlike mostof the existing haptic sculpting systems [MQW01, RE99,ST99], we use subdivision surfaces as the underlying geometricrepresentation with a physically-based force modelfor deforming the models. The turtle in Figure 1 was createdand texture-painted by ArtNova. Notice the shell pattern onthe back, and the mottling on the legs and face.In this paper, we present ArtNova, an integrated systemfor 3D texture painting and multiresolution modeling withhaptic interface and user-centric viewing. Our system hasthe following characteristics:• Interactive 3D Texture Painting – Given true 3D interactionvia a force feedback device, we can interactivelyapply predefined textures directly onto the surfacesof the model without object registration problems,in addition to painting monochrome colors.• Dynamically Adjusted Viewing – Viewpoint locationshave a direct impact on the quality of the graphical displayaccompanying haptic editing. In addition to thetypical 3D object grabbing capability, our system offersautomatic repositioning of the object to place a selectedpoint near the center of the field of view, without theuser having to switch between haptic editing and camerarepositioning. Also, it provides incremental viewpointnavigation based on the user’s gestures to provideproper views of the regions under haptic manipulation.A188

• Physically-based Force Response – A spring-basedforce model is used to emulate the surface tension whenthe user is pulling or pushing to edit the subdivisionmeshes.Several users have been able to use ArtNova to create interestingmodels with just a few minutes of training. In additionto painting with monochrome colors [GEL00], it hasalso been used to apply textures onto a 3D model’s surface aswell. The resulting meshes can be used directly for renderingand simulation without any format conversion. Preliminaryuser feedback suggests promising potentials of hapticinterfaces for 3D painting and modeling.1.2 Related WorkHaptic Interaction: Several real-time virtual environmentsystems have incorporated a haptic interface to enhancethe user’s ability to perform interaction tasks [HCT + 97,MRF + 96, MS94, RKK97]. Gibson [Gib95] and Avila andSobierajski [AS96] have proposed algorithms for objectmanipulation including haptic interaction with volumetricobjects and physically-realistic modeling of object interactions.Recently, SensAble Technologies developed theF reeF orm TM modeling system to create and explore 3Dforms using volumetric representations [ST99].Geometric Modeling: There is an abundant wealth of literatureon geometric modeling, interactive model editing,and deformation methods applied to curves and surfaces.Geometric formulations for model editing can be classifiedas pure-geometric representations such as NURBS [PT97],free-form deformation (FFD) [SP86], or physically-basedmodeling techniques such as D-NURBS [QT96]. Witha similar mathematical framework to hierarchical editing[FB88], subdivision methods allow modeling of arbitrarytopology surfaces [SZ98], while supporting multiresolutionediting [DKT98, HDD + 94, KS99, SZMS98, ZSS97]. Thereare also other sculpting techniques based on volumetricmodeling methods [GH91, MQW01, RE99, PF01]. Thehaptic modeling system inTouch uses subdivision surfacesas its underlying geometric representation [GEL00], but itlacks a physically-based force model to generate a realisticfeedback sensation.3D Texture Painting: By using standard graphics hardwareto map the brush from screen space to texture space, Hanrahanet al. allowed the user to paint directly onto the modelinstead of into texture space [HH90]. This approach hasbeen applied to scanned surfaces using 3D input devices,such as data gloves and a Polhemus tracker [ABL95]. However,the painting style of both systems can be awkward, dueeither to the difficulty in rotating an object for proper viewingduring painting, or to the deviation in paint location introducedby the registration process.General texture mapping approaches, such as [BVIG91,MYV93], are powerful but require user input to generatethe map. The approaches used in the Chameleon system[IC01] are more appropriate for casual, quick painting andnot for highly detailed models. There are also commercial23D painting systems [Hem00, COR00]. Most of them oftenuse awkward and non-intuitive mechanisms for mapping 2Dtextures onto 3D objects, or require that a texture for themodel be provided. None offers the natural painting styledesired by artists and designers.Johnson et al. introduced a method for painting a texturemap directly onto a trimmed NURBS model using a hapticinterface [JTK + 99]. Its simplicity and intuitive interfacesupport a natural painting style. However, its parameterizationtechnique is limited to NURBS and does not apply topolygonal meshes, which are more commonly encounteredin computer graphics and animation.Our approach for haptic painting is similar to that presentedin [GEL00] which can only paint colors. Our texturepainting bears some resemblance to lapped textures[PFH00]. That work determines the orientation of the texturefor overlapping patches based on user-specified vectorfields, whereas ours applies textures interactively to a localregion by brush strokes, with the orientation of the texturedetermined directly by the stroke.Camera Placement: Camera control is a fundamental problemfor 3D graphics applications. Several techniques on userinterfaces for camera control have been proposed, includingorbiting techniques mapping 2D motion into 3D interaction[CMS88, PBG92, Wer94, ZF99], use of image plane constraints[GW92, PFC + 97], and direct camera manipulationusing a 6DOF input device [WO90]. Our approach differsfrom many of the existing techniques that use 2D input devicesto directly manipulate the viewpoint. Our main focusin ArtNova is to achieve automatic placement of viewpointvia implicit control based on the user’s manipulation of thehaptic device. Our camera positioning techniques are alsouseful for touch-enabled exploration of predefined scenesand massive models [OL01].1.3 OrganizationThe rest of the paper is organized as follows. Section 2 givesan overview of our system and its user interface. We presentour new texture painting algorithm in Section 3. Multiresolutionmodeling based on subdivision surfaces with a springbasedforce computation is presented in Section 4. Section 5describes a novel feature for dynamically adjusting the viewpoint,as the objects are manipulated. We briefly highlightthe implementation of our prototype system with a hapticinterface and demonstrate its features via the actual artisticcreations of several users in Section 6.2 OverviewIn this section we give an overview of the system architectureand the user interface of our system.2.1 System ArchitectureThe overall system consists of a haptic server and a graphicalclient application, connected using VRPN [VRPN], a libraryfor distributed virtual reality applications. As with inTouch,a copy of the model is retained on both the haptic serverA189

and graphical client, and calculations that deform the modelare duplicated on both applications, so that the changes inposition of numerous vertices need not be passed over thenetwork. An overview of the system architecture is given inFigure 2.HAPTIC SERVERMultiresolutionMeshGeometryEdit ops.Contact ForceModelCollision DetectionHaptic SimulationDeformationForce ModelForceKinematicsHapticDeviceMultiresolutionMeshnetworkGeometryEdit ops.UpdatesViewing ToolsApplication UIPainting2.2 User InterfaceDeformationTexture PaintingCLIENT APPLICATIONGeometryFigure 2. System Architecture.GraphicalDisplayArtNova allows the user to edit the geometry and the surfaceappearance of a model by sculpting and painting with a hapticinterface. The user sees the model being edited, the toolbeing used, and a menu that can be operated using either thehaptic tool or a mouse. Each type of model manipulation isindicated by a different tool. For example, the user movesthe object with a mechanical claw, paints with a paintbrush,and deforms with a suction cup.As an alternative to the claw tool for moving the object,the user’s viewpoint can be changed using the viewing techniquesdescribed in Sec. 5. An automatic repositioning featurelets the user move the last touched point on the model tothe center of the viewing area using a single keystroke, andthere is a “flying mode” controlled by the haptic device.For painting there are a continuous color picker, slidersfor brush width and falloff of paint opacity, and choice oftextures for texture painting. The width of the stroke can alsobe changed by adjusting the pressure applied when painting.A basic undo feature is provided for deformations andpainting, and there are provisions for saving models andscreen shots. A snapshot of the user interface is shown inFigure 3.3 Texture PaintingIn addition to modeling, ArtNova allows the user to painttextures and colors onto the model using a virtual paintbrush.Arbitrary polygonal models can be painted, and eachstroke has a configurable falloff, fading into the background3Figure 3. The graphical user interface. The useris performing a deformation on a painted toroidalbase mesh.near the boundaries of the stroke. The detailed algorithm forpainting monochrome colors was described in [GEL00]. Webriefly summarize it here before describing our novel texturepainting method.3.1 Painting AlgorithmEach stroke of the brush is decomposed into a sequence ofstroke segments, which are represented as 3-space vectorslinking the positions of the tool tip at successive frames. Thebrush radius is computed at the stroke endpoints (based onthe force exerted by the user) and linearly interpolated acrossthe length of the vector. This radius determines a volume ofinfluence in 3D. Triangles are painted starting with the onecontaining the tail of the stroke segment. When one trianglehas been painted, any neighbors that also extend into thevolume of influence are then painted.We paint each triangle by rasterizing the correspondingtriangle in texture space. As the triangle is rasterized, foreach texel p t we determine the corresponding point p w inworld space on the surface of the model. We then calculateD asD = ‖p w − q‖,R s (q)where q is the point on the stroke segment nearest p w , andR s (q) is the stroke radius at q. Once we have D, we computethe color of p t by blending the color being painted with thebackground according to a falloff function depending on D.Because we use a straight vector to represent a portionof a stroke that actually follows a curved surface, there canbe artifacts if the surface curvature relative to the length ofthe stroke segment is appreciable. Typically, however, thestroke segments are quite short and this problem does notarise.3.2 Texture-Mapped PaintTexture-mapped paint builds upon our algorithm for paintingmonochrome colors. For clarity we will refer to theA190

texture map into which colors are drawn as the target texturemap, and the predefined texture array as the source texturemap. For each texel being rasterized in the target texturemap, we compute the corresponding point p w in worldspace, and its nearest neighbor q on the stroke segment (seeFigure 4). We then compute two coordinates s and t that willbe used to look up colors in the source texture map. The scoordinate represents length along the stroke, and t representssigned distance from the stroke. We compute s and tas follows: Let the current stroke segment be represented bya vector ⃗v with tail p v . Then, for s, we maintain the totallength of all previous segments of a given stroke, and addthe distance ‖q − p v ‖. The sign of t is equal to the sign of((p w − p v ) × ⃗v) · ⃗nwhere ⃗n is the stored normal vector of the triangle containingp w . The magnitude of t is just ‖p w − q‖.p w¡ ¡¡¡¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡¢ ¡¡¡¡¡¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡ ¡ ¡¡¡¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡¢ ¡¡¡¡¡¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡ ¡ ¡¡¡¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡¢ ¡¡¡¡¡¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡¢¡ Source Texture MapWorld Spaceqp tTarget Texture MapFigure 4. Texture painting.Before s and t are used to look up a color, they are scaledby a user-adjustable factor so that the texture features willbe the desired size on the model. Texture lookup is performedmodulo the dimensions of the texture patch, so it isimportant that the texture be periodic. Note that, althoughthe texture will repeat along a stroke, the user can break upthe periodicity by starting new strokes and altering strokeorientation.At the boundary between stroke segments, the point q onthe stroke vector nearest to p w may be an endpoint. If q isthe tail end of the stroke segment, texture is not applied forthat point. If q is the front, then t is still taken as the distancefrom p w to q, with sign given as above, while s is fixed. Thishas the effect of replicating pixels from the trailing stroke tofill gaps between roughly rectangular strips. Figure 5 showsan example of a textured cow model.3.3 Accelerated Texture PaintingThe original algorithms for color and texture painting rasterizedthe color data into the texture memory in software. We4Figure 5. Cuts of beef indicated by different texturesusing ArtNova.have developed an improved algorithm that relies on moderngraphics hardware to copy the texture. For each strokesegment, we first accumulate a list of triangles that extendinto the volume of influence of the segment. Then the regionof the texture parameter domain covered by those trianglesis rendered with a fixed z coordinate in an orthogonalprojection. If there are multiple patches in the target texture,the relevant triangles from each patch are rendered intoa different region of the frame buffer. These triangles aretextured, using coordinates generated dynamically that indexinto the source texture memory. The resulting image isthen copied into the target texture memory with alpha blending.To produce the appropriate falloff function, the sourcetexture has been prepared with the alpha values determinedby the falloff. We can paint simple colors by choosing amonochromatic source texture.3.4 Texture CoordinatesThe above algorithms assume that texture coordinates for alltriangles in the model have been determined. If they are notprovided, we use a simple algorithm for generating texturecoordinates. Since the user is painting on the model, not thetexture map, we need not concern ourselves with contiguityof the image in texture memory. Also, there is relatively littleconcern with distortion, for the same reason. If the choiceof texture coordinates distorts the image, the distorted versionof the image will appear in the texture memory, not onthe model. The algorithm we currently use groups trianglesinto pairs, which are mapped to squares in texture space.The areas of the squares are chosen to approximately reflectthe initial areas of the triangles.3.5 Comparison with other approaches.One sophisticated program for painting on 3D models isDeepPaint [Hem00], a commercial product distributed byRight Hemisphere. Its interaction is fundamentally two dimensional,but the geometric model for applying a textureto the surface can be compared as an issue on its own. DeepA191

Paint offers two modes for texture painting. In one, the textureis copied directly to the texture memory, so that distortionsin the parameterization are reflected in the appearanceof texturing. For instance, the scale of the texture may befiner near the “north pole” of a sphere. In the other mode, thetexture is applied in screen space, and then remapped as itis transferred to texture memory. This is the same approachthat Chameleon [IC01] uses to apply solid color. This approachyields results that are independent of the parameterizationof the object being painted, but it creates a distinctive“stretching” of the texture near the silhouette of the figure,which becomes noticeable when object is rotated to a differentposition. Because our approach determines temporarytexture coordinates locally in the neighborhood of a portionof a single stroke, the overall scale of the texture is not dependenton where it is placed, making the painting more predictablefor the user.4 Multiresolution Mesh EditingOur model editor is based on inTouch’s geometric framework[GEL00], which is strongly influenced by Zorin etal. [ZSS97]. For completeness, we briefly describe theframework here and then describe the new simplified forcemodel used during deformation in ArtNova.4.1 The Mesh Data StructureWe use a subdivision framework to represent our geometry.We store a coarse, triangular base mesh M 0 and severalmeshes at finer resolutions M i (i > 0). By a single stageof Loop subdivision, each mesh M i uniquely determines afiner mesh Mi+1 sub.M i+1 sub is used as a reference mesh for thedefinition of M i+1 . Every vertex in the actual mesh M i+1corresponds to a vertex of Mi+1 sub , but differs from it by a displacementvector stored with the vertex. In this way we canchoose to edit at a specific resolution by moving vertices of agiven mesh M i . Vertices at finer levels retain their displacementvectors and are thus carried along by the motion of thesubdivided surface.In principle we could modify M i without changing M i−1at all, since the vertices of M i are different from the verticesof Misub (gotten by subdividing M i−1 ). However, wealso perform a smoothing step using a method given byTaubin [Tau95] to modify coarser levels. In this way, forinstance, an accumulation of edits at a high resolution, alltending to raise up one side of the model, can result in arepositioning of the coarser level vertices to better reflectthe new overall geometry of the model.4.2 Deformation AlgorithmsTo edit the model, the user simply places the tool against themodel, presses the PHANTOM button, and moves the tool.As the surface is edited, the user can feel a resisting forceand see the surface deform. The edit resolution (the choiceof mesh M i to modify directly) is presented to the user as a“bump size.”54.2.1 Surface ModificationSurface deformation is performed by moving a single triangleof the edit mesh M i . When the user begins a deformation,the point of contact with the surface determines aunique triangle at the edit resolution, and a unique referencepoint on that triangle. For each frame, the successive positionsof the tool tip define a motion vector ⃗m, which is usedto move the three vertices of the selected triangle. Each vertexis moved in the direction of ⃗m, and by a distance scaledso that vertices nearer the current point of the tool tip aremoved farthest. More precisely, the distance d i from thereference point to each vertex v i is computed, and the movementvector ⃗m i for each vertex is given by⃗m i =4.2.2 Force Model(1 −)d i⃗m.d 0 + d 1 + d 2When the user places the tool against the model, there isa restoring force generated by the haptic rendering library,based on collision information from H-Collide [GLGT00].When the user begins deforming the surface, the restoringforces are turned off, and the initial 3-space location of thetool tip, p 0 , is recorded. The user is then free to move thetool in any direction. To provide feedback, a Hooke’s lawspring force is established between the tool tip and p 0 , givenbyf = −k(p tip − p 0 ),where p tip is the location of the tool tip and k is a smallspring constant.The spring constant is chosen so that the user can movethe tip a sizable distance in screen space before the forcebecomes a substantial hindrance. When the user releases thebutton, the spring force is turned off and the usual restoringforces are turned on with the surface in its new position.Because our force model is based on the initial positionof the tool, the force computation is decoupled from the positionof the surface. This provides a smoother feel to theuser than computing the force from the instantaneous distanceto the surface that is being moved, because computingthe surface deformation is much slower than the 1 kHz hapticresponse rate.5 Dynamic Viewing TechniquesAs the user performs painting or modeling tasks over theentire model, the user will need to edit back-facing portionsof the model from time to time. Typically the user repositionsthe model by performing a “grab and turn” operationusing the application. We have developed several noveluser-centric viewing techniques that make this task easierand more efficient. In addition to using the force feedbackdevice for haptic manipulation, we also use it implicitly, andsimultaneously, as a mechanism for viewpoint placement.These techniques allow users to express their intentions in anatural way, with a minimum of switching between editingand changing the view.A192

Our approach to the problem is to reposition the camerabased on the configuration (i.e. position and orientation) ofthe haptic probe so that the region of interest on the modelsurface is placed at the center of the view. We call our techniques“user-centric” because the haptic tool indicates wherethe user wants to view the object from, rather than where theobject should be.5.1 Grabbing OperationIn the typical grabbing operation [RH92], the object ismoved preserving the transformation between the virtualprobe and the grabbing point. At the instant of grabbing, twopoints are picked: A, a point in the object, and B, the currentposition of the virtual probe. The transformation T BAfrom the probe to the object is set constant as the virtualprobe moves. Therefore, a displacement of the probe, ∆T B ,produces a displacement of the object, ∆T g .5.3 Automatic RepositioningAnother novel functionality called automatic repositioningallows the user to reorient the model quickly without interruptingthe work flow to change tools. When the automaticrepositioning key is pressed, the last point touched on themodel is taken to define the region of interest. A transformationT r is computed that moves the camera to a locationalong the normal direction of the object at the point of interest(as shown in Fig. 7). Then, we apply the inverse transformationto the object incrementally, so that the object appearsto rotate to the new position.T BA = T A T B −1 ,∆T g = T BA ∆T B T BA −1 .where T A and T B are the transformations that represent thelocation and orientation of the object and the virtual probe.5.2 Viewpoint NavigationWe introduce a new concept called “viewpoint navigation”.This functionality is based on moving the viewpoint aroundthe object to be modeled or painted, based on the gestures ofthe user. The position and orientation of the haptic device ateach time are used as “commands”, which set a transformation,T n , on the position of the virtual probe and the camera,as shown in Fig. 6.In our system we apply the inverse transformation to theobject that is being manipulated, producing the same visualeffect. In addition, we apply this transformation incrementally,whenever the viewpoint navigation is enabled.∆T n = K ∆T H .where ∆T H represents the variation of the position and orientationof the haptic device.Figure 6. Viewpoint navigation based on user’sintentions.6Figure 7. Automatic repositioning of viewpoint.5.4 Combining all the FunctionalitiesThe transformation that sets the location and orientation ofthe object is updated every time step for better viewing toaid the model manipulation and editing tasks, depending onwhich functionality is enabled. We have:T A,k+1 = ∆T g T A,k , if grabbing is enabled;T A,k+1 = ∆T −1 n T A,k , if viewpoint navigation is enabled;T A,k+1 = ∆T −1 r T A,k , if automatic repositioning is enabled.6 ResultsWe have designed and implemented ArtNova as a proof-ofconceptprototype system. In this section, we demonstratethe system capability and discuss issues related to the userinterface design.6.1 Prototype DemonstrationWe use a dual-processor Pentium III PC as a haptic server,a SensAble Technologies’ PHANTOM as a haptic device, aSilicon Graphics Inc. R12000 Infinite Reality for rendering,a large screen with a back projection system for graphicaldisplay, and UNC’s VRPN library [VRPN] for a networktransparentinterface between application programs and ourhaptic system. The system is written in C++ using theOpenGL and GLUT libraries. However, the design frameworkof our system is applicable to all types of haptic devicesand libraries, as well as graphical display and computingplatforms.Several users with little or no experience in using modelingor painting systems were able to create interesting modelsusing ArtNova with little training. Each user was givenA193

Figure 8. A tree.a selection of simple base meshes of various shapes (e.g.spherical, toroidal, etc.) and light gray in color. A few examplesof their art work are given in Figures 1, 3, 8, and 9.The accompanying videotape illustrates a few examplesof the modeling and painting operations performed usingArtNova. Along with some more images of the models createdby our users, the video clips are also available at:http://www.cs.unc.edu/∼geom/ArtNova.6.2 User ExperiencesWe have asked several users to evaluate our system by creatingand texture-painting a few models. We briefly summarizesome of their comments here:• The spring-based force model feels natural. Comparingthe experiences of performing model deformation withand without the physically-based force model, most ofthe users found the newly added force model to be animprovement.• The users commented that the force feedback was usefulin detecting and maintaining contact with the modelsurfaces, when performing highly detailed painting.• Users also felt that texture painting was easy to use, andhad little trouble getting visually pleasing results.• Users found automatic viewpoint adjustment and repositioningto be intuitive and natural.• The overall graphical user interface with our new 3Dtool metaphor was found to be intuitive and easy to understand.77 SummaryFigure 9. A fish.We have presented an integrated system for 3D texture paintingand multiresolution modeling with a haptic interface anduser-centric viewing. An artist or a designer can use this systemto create and refine a three-dimensional multiresolutionpolygonal mesh, and further enhance its appearance by directlypainting textures onto its surface. The system allowsusers to naturally create complex forms and patterns aidednot only by visual feedback but also by their sense of touch.Based on preliminary user feedback, we believe these featuresconsiderably improve the ease and expressiveness of3D modeling and texture painting. We are currently planningan extensive, formal user study to carefully evaluateand assess the contribution of various elements of this systemon enhancing users’ experiences in digital model design.8 AcknowledgmentsThis research is supported in part by a fellowship of the Governmentof the Basque Country, NSF DMI-9900157, NSFIIS-9821067, ONR N00014-01-1-0067 and Intel. We wouldlike to thank the following individuals for their suggestionand assistance: Arthur Gregory, Stephen Ehmann, MichaelRosenthal, Adrian Ilie, Sarah Hoff, Bryan Crumpler, DerekHartman, Scott Cooper, and Joohi Lee.References[ABL95][AS96][BVIG91][CMS88]Maneesh Agrawala, Andrew C. Beers, and Marc Levoy. 3Dpainting on scanned surfaces. In Pat Hanrahan and JimWinget, editors, 1995 Symposium on Interactive 3D Graphics,pages 145–150. ACM SIGGRAPH, April 1995.R. S. Avila and L. M. Sobierajski. A haptic interaction methodfor volume visualization. Proceedings of Visualization’96,pages 197–204, 1996.Chakib Bennis, Jean-Marc Vézien, Gérard Iglésias, and AndréGagalowicz. Piecewise surface flattening for non-distortedtexture mapping. In Thomas W. Sederberg, editor, ComputerGraphics (SIGGRAPH ’91 Proceedings), volume 25, pages237–246, July 1991.Michael Chen, S. Joy Mountford, and Abigail Sellen. A studyin interactive 3-D rotation using 2-D control devices. In JohnDill, editor, Computer Graphics (SIGGRAPH ’88 Proceedings),volume 22, pages 121–129, August 1988.A194

[COR00][DKT98][FB88][GEL00][GH91][Gib95][GLGT00][GW92]COREL. Painter.http://newgraphics.corel.com/products/painter6.html, 2000.T. DeRose, M. Kass, and T. Troung. Subdivision surfaces incharacter animation. Proc. of ACM SIGGRAPH, 1998.D. Forsey and R.H. Bartels. Heirarchical B-spline refinement.In Proc. of ACM Siggraph, pages 205–212, 1988.A. Gregory, S. Ehmann, and M. C. Lin. inTouch: Interactivemultiresolution modeling and 3d painting with a haptic interface.Proc. of IEEE VR Conference, 2000.Tinsley A. Galyean and John F. Hughes. Sculpting: An interactivevolumetric modeling technique. In Thomas W. Sederberg,editor, Computer Graphics (SIGGRAPH ’91 Proceedings),volume 25, pages 267–274, July 1991.S. Gibson. Beyond volume rendering: Visualization, hapticexploration, and physical modeling of element-based objects.In Proc. Eurographics workshop on Visualization in ScientificComputing, pages 10–24, 1995.A. Gregory, M. Lin, S. Gottschalk, and R. Taylor. Real-timecollision detection for haptic interaction using a 3-dof forcefeedback device. Computational Geometry: Theory and Applications,Special Issue on Virtual Environments, 15(1-3):pp.69–89, February 2000.Michael Gleicher and Andrew Witkin. Through-the-lens cameracontrol. In Edwin E. Catmull, editor, Computer Graphics(SIGGRAPH ’92 Proceedings), volume 26, pages 331–340,July 1992.[HCT + 97] J. Hollerbach, E. Cohen, W. Thompson, R. Freier, D. Johnson,A. Nahvi, D. Nelson, and T. Thompson II. Haptic interfacingfor virtual prototyping of mechanical CAD designs. CDROMProc. of ASME Design for Manufacturing Symposium, 1997.[HDD + 94] H. Hoppe, T. DeRose, T. Duchamp, M. Halstead, H. Jin,J. McDonald, J. Schweitzer, and W. Stuetzle. Piecewisesmooth surface reconstruction. In Proceedings of ACM SIG-GRAPH, pages 295–302, 1994.[Hem00][HH90][IC01][JTK + 99][KS99][Mas98][MQW01]Right Hemisphere. Deep paint. http://www.us.deeppaint.com/,2000.Pat Hanrahan and Paul E. Haeberli. Direct WYSIWYG paintingand texturing on 3D shapes. In Forest Baskett, editor, ComputerGraphics (SIGGRAPH ’90 Proceedings), volume 24,pages 215–223, August 1990.T. Igarashi and D. Cosgrove. Adaptive unwrapping for interactivetexture painting. Proc. of ACM Symposium on Interactive3D Graphics, pages 209–216, 2001.D. Johnson, T. V. Thompson II, M. Kaplan, D. Nelson, andE. Cohen. Painting textures with a haptic interface. Proceedingsof IEEE Virtual Reality Conference, 1999.A. Khodakovsky and P. Schröder. Fine level feature editingfor subdivision surfaces. Proceedings of ACM Symposium onSolid Modeling and Applications, 1999.Thomas Massie. A tangible goal for 3D modeling. IEEE ComputerGraphics and Applications, May/June, 1998.K. McDonnell, H. Qin, and R. Wlodarczyk. Virtual clay: Areal-time sculpting system with haptic interface. Proc. of ACMSymposium on Interactive 3D Graphics, pages 179–190, 2001.[MRF + 96] William Mark, Scott Randolph, Mark Finch, James Van Verth,and Russell M. Taylor II. Adding force feedback to graphicssystems: Issues and solutions. In Holly Rushmeier, editor,SIGGRAPH 96 Conference Proceedings, Annual ConferenceSeries, pages 447–452, 1996.[MS94]T. M. Massie and J. K. Salisbury. The phantom haptic interface:A device for probing virtual objects. Proc. of ASMEHaptic Interfaces for Virtual Environment and TeleoperatorSystems, 1:295–301, 1994.8[MYV93][OL01]Jérôme Maillot, Hussein Yahia, and Anne Verroust. Interactivetexture mapping. In James T. Kajiya, editor, Computer Graphics(SIGGRAPH ’93 Proceedings), volume 27, pages 27–34,August 1993.M. Otaduy and M. Lin. User-centric viewpoint computationfor haptic exploration and manipulation. Proc. of IEEE Visualization,2001. To appear.[PBG92] C. Phillips, N. Badler, and J. Granieri. Automatic viewingcontrol for 3D direct manipulation. Proc. of ACM Symposiumon Interactive 3D Graphics, pages 71–74, 1992.[PF01][PFC + 97]R. Perry and S. Friskin. Kizamu: A system for sculpting digitalcharacters. Computer Graphics (ACM SIGGRAPH’01),2001.J. Pierce, A. Forsberg, M. Conway, S. Hong, R. Zeleznik, andM. Mine. Image plane interaction techniques in 3D immersiveenvironments. Proc. of ACM Symposium on Interactive 3DGraphics, pages 39–44, 1997.[PFH00] E. Praun, A. Finkelstein, and H. Hoppe. Lapped textures.Proc. of ACM SIGGRAPH, pages 465–470, 2000.[PT97][QT96][RE99][RH92][RKK97][SP86]L. A. Piegl and W. Tiller. The NURBS Book. Springer Verlag,1997. 2nd Edition.Hong Qin and Demetri Terzopoulos. D-NURBS: A physics-Based framework for geometric design. IEEE Transactionson Visualization and Computer Graphics, 2(1):85–96, March1996. ISSN 1077-2626.A. Raviv and G. Elber. Three dimensional freeform sculptingvia zero sets of scalar trivariate functions. ACM Symposiumon Solid Modeling and Applications, 1999.Warren Robinett and Richard Holloway. Implementation offlying, scaling, and grabbing in virtual worlds. In DavidZeltzer, editor, Computer Graphics (1992 Symposium on Interactive3D Graphics), volume 25, pages 189–192, March1992.D.C. Ruspini, K. Kolarov, and O. Khatib. The haptic display ofcomplex graphical environments. Proc. of ACM SIGGRAPH,pages 345–352, 1997.Thomas W. Sederberg and Scott R. Parry. Free-form deformationof solid geometric models. In David C. Evans andRussell J. Athay, editors, Computer Graphics (SIGGRAPH ’86Proceedings), volume 20, pages 151–160, August 1986.[ST99] SensAble Technologies Inc. freeform T M modeling system.http://www.sensable.com/freeform, 1999.[SZ98][SZMS98]P. Schröder and D. Zorin. Subdivision for modeling and animation.ACM SIGGRAPH Course Notes, 1998.T. Sederberg, J. Zheng, M.Sabin, and D. Sewell. Non-uniformrecursive subdivision surfaces. Computer Graphics (ACMSIGGRAPH’98), 1998.[Tau95] Gabriel Taubin. A signal processing approach to fair surfacedesign. In Robert Cook, editor, SIGGRAPH 95 ConferenceProceedings, Annual Conference Series, pages 351–358.ACM SIGGRAPH, Addison Wesley, August 1995. held in LosAngeles, California, 06-11 August 1995.[VRPN]Virtual Reality Peripheral Network.http://www.cs.unc.edu/research/nano/manual/vrpn.[Wer94] Josie Wernecke. The Inventor Mentor. Addison-Wesley, 1994.[WO90][ZF99]C. Ware and S. Osborne. Exploration and virtual camera controlin virtual three dimensional environments. Proc. of ACMSymposium on Interactive 3D Graphics, pages 175–183, 1990.Robert Zeleznik and A. Forsberg. Unicam 2D gestural cameracontrols for 3D environments. Proc. of ACM Symposium onInteractive 3D Graphics, pages 169–173, 1999.[ZSS97] D. Zorin, P. Schröder, and W. Sweldens. Interactive multiresolutionmesh editing. Computer Graphics (ACM SIG-GRAPH’97), 1997.A195

RECENT ADVANCES IN HAPTIC RENDERINGAND APPLICATIONSSupplementary Course NotesPART II – Presentation SlidesMing C. LinUniversity of North Carolina at Chapel HillMiguel A. OtaduyETH ZurichB1

Course: Recent Advances in Haptic Rendering and ApplicationsDefinitionIntroduction and OverviewHaptic Rendering: display of informationthrough force or tactile cues.Ming C. LinUNC-Chapel HillMiguel A. OtaduyETH-ZurichHaptic RenderingVisual Display‣ Intuitive interactionwith virtual environments‣ Increase presencehttp://www.cs.unc.edu/~linlin@cs.unc.eduhttp://graphics.ethz.ch/~otmiguelotaduy@inf.ethz.chAuditory FeedbackHistoryHistory• Used in master-slave telerobotic systems• First suggested by Sutherland [1965] forinteraction with VEs• Project GROPE in UNC:– 2D force field simulation [1971]– 3D molecular docking [1990]• [Minsky et al. 1990]: rendering of 2D textures.2D height fieldCompute F x , F yforces usinggradients2DOF deviceHistory• [Zilles and Salisbury 1995; Ruspini et al.1997]: point-object interaction.xHistory• [McNeely et al. 1999]: interaction between rigidobjects. Also called 6DOF haptic rendering.pConstrain x to thesurfaceForce given bydist(x,p)Per-sample collisiontestNet force and torqueObject, point (p)and proxy (x)3DOF deviceSample objects6DOF deviceB3

Description of the ProblemDescription of the Problem1. Move grabbed object according to input2. Compute contact forces between objects3. Apply net force and torque to grabbed objectSimilar to Rigid Body Simulation• Control engineering analysis:Pos*PosUserControlF 1 F 2F*HapticActuatorsRenderingFDevice +BiomechanicsPosDescription of the ProblemDescription of the Problem• Control engineering analysis:• Control engineering analysis:Pos*PosactiveUserControlF 1 F 2F*HapticActuatorsRenderingFDevice +BiomechanicsPosPos*PospassiveUserControlF 1 F 2F*HapticActuatorsRenderingFDevice +BiomechanicsHuman-in-the-loopPosDescription of the ProblemDescription of the Problem• Control engineering analysis:Pos*PosUserControlF 1 F 2F*HapticActuatorsRenderingFPos• 2 major components:– Collision detection / contact determination– Contact response model and force computationDevice +BiomechanicsHuman-in-the-loop• High sensitivity to instabilities!!• High update rates required!! (kHz for high stiffness)B4

Course: Recent Advances in Haptic Rendering and ApplicationsSession I: Design Guidelines and Basic Point-Based TechniquesHaptic Perception andImplications for DesignRoberta KlatzkyCarnegie Mellon UniversityDept. of PsychologyHuman-Computer Interaction InstituteCenter for the Neural Basis of CognitionHaptic Perceptionand implications for design• What is haptic perception?• Based on touch receptors• in skin, muscles, tendons, joints• Usually involves active seeking ofinformation (exploration)Outline of Talk• Neural Coding of Touch Primitives• Functions of Peripheral Receptors• Haptic Primitives and their Encoding• Complementary Functions of Haptics andVision• Emergent Issues for Design1. Neural Coding of TouchPrimitives• Touch Receptors– Mechanoreceptors and their Function– Other skin receptors: thermal, pain• Pathways from receptors to brainTouch ReceptorsClasses of Receptors• Touch sensations are mediated by receptorsthat respond to pressure, vibration, and heat flow.• The receptors are found in two regions:-- Within skin: cutaneous sensingpressure, temperature, pain-- Beneath skin in muscles, tendons, joints:kinesthetic sensinglimb position and movement• Receptors that respond to pressure and vibration are calledmechanoreceptors• Mechanoreceptors are found in skin (cutaneous) and muscles,tendons, and joints (kinesthesis)• The skin also includes other receptors– that signal skin warming and cooling (thermo-receptors)– that signal pain (nociceptors)B5

Skin Mechanoreceptors have specialized endingsFunctional characteristics of Skin Mechanoreceptors:Receptive field size and adaptation rateFA I SA I FA II SA IIFunctional Name:ReceptorsMeissnerFast adapting Type IMerkelSlow adapting Type IMeissner’s Merkel Pacinian RuffiniCorpuscle Cell Complex Corpuscle EndingRuffiniSlow adapting Type IIReceptiveFieldIntensity and Time Course of Neural Signal (adaptation)NeuralSpike trainJohannson & Valbo, 1983PacinianFast adapting Type IIKandel et. al., 2000StimulusFrequency Sensitivity:One-channel-per-mechanoreceptor modelThermal ReceptorsAmplituderequiredfor athresholdresponse,at a givenfrequency(dBrelative to1 µm)Thermo-receptors lack specialized endings.Two populations of nerve fibers: warm and coldEach has its own response range, with some overlap.Note: body temperature in Centigrade = 37º.NeuralresponserateColdWarmBolanowski et al., 1988Kenshalo, 197620 25 30 35 40 45 50Temperature ( C )A variety of neural fibers leadfrom periphery to spinal cord.Two pathways from cord to the brain4 types, varying in conduction speed and associated receptorpopulationsFast, newSlow, oldFiber Speed Associated receptorsA-α fast kinestheticA-β moderately fast kinesthesthetic & cutaneous mechano.A-δ moderately fast thermo- & nociceptor (sharp pain)C slow thermo- & nociceptor (burning pain)Unidentified sourceCutaneous,kinestheticNociceptor, thermal,some mechanoreceptorsB6

The brain: Primary and SecondarySomatosensory CortexSomatosensory “Homunculus” in SIresults from somatotopic mapping*MotorSISIIAreas 5,7SI includes Brodmann areas:3a: Muscle3b: Skin (SA and FA)1: Skin (FA)2: Pressure, joints*Adjacent on skin⇒Adjacent in SISI projects to SII, 5, 7SIThere are severalsuch maps.SIIPenfield & Rasmussan, 1950Somatosensory areas in brain are plastic:Re-assignment of receptive fields afteramputation of a digitAreas in SI thatonce respondedto 3rd fingertipare now activatedby finger 2 and 4,plus base of 32. Functions of peripheralreceptors• Cutaneous mechanoreceptors providearray (tactile) sensing.• Kinesthetic mechanoreceptors provide asense of limb position, movement.• They support different human abilities.Merzenich et al., 1984Grasping is supported byFAI mechanoreceptors thatdetect incipient slipGrasp, lift… ReturnUnder near slip conditionsTactile Pattern Perception isbased on SA I mechanoreceptorsSpatial plot of the response of a skin mechanoreceptor to a Braille patternswept through its receptive field: Each tick mark is a neural impulse givencontact from that stimulus location. SA I have resolution of ~.5 mm at fingertip.Load Force NGrip Force NPosition mmGrip: LoadratioTimeSlip thresholdLoad Force NGrip Force NPosition mmForce RatioFA ISA IFA IISA IIInputpatternResponseSA IFA ISA IIFA IIPatternpreservationJohansson & Westling,1984; Westling, 1986Phillips, Johansson & Johnson, 1990B7

Curvature perception alsoreflects SA I responsesData are shown for 7 curves, ranging from radius zero to radius 1.44 mmRoughness perception is directlyrelated to spatial variation in SA IresponsesMean impulse rate does not predict roughness magnitude.SAI RA PCMeanSA IResponseSubjectiveRoughnessMagnitude0 25 50 75 100 0 25 50 75 100 0 25 50 75 100Mean impulse rate (impulses/s)Note: largeroughnessrange for asingle mean-4 -2 0 2 4Distance of receptive field from center of indentation (mm)0 25 50 75 100 0 20 40 80 0 25 50Spatial variation in impulse rate (impulses/s)Goodwin et al., 1995Connor, Hsaio, Phillips, Johnson, 1990What do PC receptors signal?Kinesthetic mechanoreceptors areinvolved in softness perceptionSoftness discrimination of rubber specimens vs. rigidly covered springs• Anything that causes deep vibrations-- Transient vibrations from contact-- Micron-element textural variationsOnly PCs respond continuously to thetextured portion of a half-textured platewith ht. 1 µm• Pure kinesthesis condition: anesthetize fingertips -- no cutaneous cues• Pure cutaneous condition: passively press stimulus against fingerpad-- no muscle/tendon/joint involvement• Dual-cue condition: normal, active touch• Deformable surface (rubber) could be judged from cutaneous cues alone.• Rigid (spring-loaded) surfaces required kinesthesis + cutaneous cues.Srinivasan, Whitehouse, & LaMotte, 1990Srinivasan & LaMotte, 19953. Haptic primitives and theirencoding• Haptically perceived properties of objectsand surfaces• Link between exploration and perception ofhaptic propertiesWhat are haptically perceptibleproperties of objects?• Geometry: Size (2-D, 3-D),Shape, Pointiness..• Surface: Roughness,Slipperiness, Warmth…• Substance: Compliance,WeightB8

How do people perceivethese properties?• Contact– Sufficient for some information• Purposive Exploration– Highly informative– Highly stereotyped– Directly linked to the information that is desiredMatch-to-sample task shows linksfrom properties to explorationTask:Pick bestmatch tostandardstimulus ongivendimensionData: HowpeopleexploredOther properties not shown: temperature, weight, hardnessLederman & Klatzky, 1987Exploratory Procedures andLinks to PropertiesOptimal and non-optimalexplorationThis explorationoptimizes output ofunderlying neuralsystems andcomputationalprocesses.Row: EP used todiscriminateColumn: PropertydiscriminatedDiagonal: PredictedEP/property linkBut,Non-optimalexploration stilldelivers someinformation.Lederman & Klatzky, 1987Lederman & Klatzky, 1987;Klatzky & Lederman, 19933. Complementary functions ofhaptics and vision• Material vs. Geometric Properties• Differential Accessibility of these Properties– Preferences for free exploration– Speed of perceptual access• Integration across modalitiesMaterial vs. Geometric Properties• Geometric Properties– Depend on particular sampled object– Describe size, structure– Relatively accessible to Vision• Material Properties– Independent of object’s form (within extremes)– Describe surface and substance– Examples: warmth, roughness, …– Relatively accessible to HapticsB9

Preferences under free explorationwith vision and touch• Subjects asked, which of two objects is rougher, harder…• Some questions were easy (roughness of sandpaper vs.silk), some difficult (size of golf ball vs. marshmallow)Touch is used fordifficult judgments ofmaterial properties.Otherwise, visionused for perceptualcomparison or totrigger memoryretrieval.Klatzky, Lederman, & Matula 1993Mean percent touched100806040200RoughHardTempWeightDimensionSizeDifficultEasyShapeSpeed of access to properties bytouch favors material overgeometry• Subjects search for a target object presentedto one finger, while other fingers get 0-5distractorsLederman & Klatzky, 1997; apparatus developed at Queen’s UniversitySpeed of access, continued• Judgments of several types of propertiesMaterialSpeed of access, continuedIncreasing slopeindicates finger byfinger search:characteristic ofspatial queriesLederman & Klatzky, 1997Edge contentSpatial RelationsLederman & Klatzky, 1997Number fingers stimulatedFlat slope indicatestarget “pops out”:characteristic ofmaterial, edgequeriesHaptic/visual integration• If material properties are relatively accessibleto touch,• And geometric properties are relativelyaccessible to vision,• How do the two modalities interact?• Apparently -- often -- optimally!– Ernst and BanksMaximum Likelihood Model• Vision, touch assumed to input to a commonintegrator• Integrator weights signals by reliability:inversely related to variance• Integrator adds weighted signalsB10

Applying the model toroughness perceptionVisual input from stereo gogglesNoise added by eliminating stereofrom some locationsHaptic input from PHANTOM forcefeedback deviceResults: Optimal weighting of vision,depending on noise5. Emergent Issues for Design• Implications of encoding pathways• Implications of intersensory integration• The “holy grail” of haptic interface designErnst & Banks, 2002Implications of hapticencoding pathways• Multiple receptors enable touch to encode a varietyof neural primitives– A map of deformation on the fingertip– Vibration, skin stretch, temperature– Also enable fine manipulation• Peripheral primitives, combined with activeexploration, richly depict objects and surfaces– Haptics favors material over geometry– Haptics and Vision are complementaryImplications of intersensoryintegration• Inputs from other sensory systems are integratedwith haptics• Integration is sensitive to sensory reliability– Vision can compensate for relatively coarse spatial codingby haptics– Haptics can compensate for relatively coarse materialcoding by vision– One sense can be “fooled” by the other,• e.g., coarse visual textures make surfaces feel rougher;• visual curves make haptic edges feel rounderThe “holy grail”• Not just your grandmother’s force-feedbackdevice• Need dense, robust, array stimulators• Thermal stimulation highly useful• Exploration should be maximally enabled• Add other modalities to capitalize on humanmulti-sensory abilitiesB11

Course: Recent Advances in Haptic Rendering and ApplicationsSession I: Design Guidelines and Basic Point-Based TechniquesIntroduction to 3-DoFHaptic RenderingMing C. LinDepartment of Computer ScienceUniversity of North Carolina at Chapel Hillhttp://www.cs.unc.edu/~linlin@cs.unc.edu3DOF Haptics: Introduction• Output: 3D force 3DOF haptics• Limited to applications where point-objectinteraction is enough.– Haptic visualization of data– Painting and sculpting– Some medical applicationsObject-objectPoint-object3DOF Haptics:Basic approach• Check if point penetrates an object.• Find closest point on the surface.• Penalty-based force.F xF = k ⋅ x3DOF Haptics: The problems• Force discontinuities when crossingboundaries of internal Voronoi cells.F 2F 1Unexpected forcediscontinuities (both inmagnitude and direction)are very disturbing!3DOF Haptics: The problems• Pop-through thin objects.motionAfter the mid line iscrossed, the force helpspopping through.3DOF Haptics:Point-to-plane• Mark et al., 1996.• For distributed applications. The simulator sendsthe equation of a plane to the haptic loop. Theupdate of the plane is asynchronous.• Forces are computed between the haptic point andthe plane.• Possible stiffness is limited, because too stiff wouldbring a jerky behavior at low update rates.B12

3DOF Haptics: God-object• Zilles and Salisbury, 1995.• Use the position of the haptic interface point(HIP) and a set of local constraint surfaces tocompute the position of god-object (GO).• Constraint surfaces defined using heuristics.• Compute GO as the point that minimizes thedistance from HIP to the constraint surfaces.Lagrange multipliers.3DOF Haptics: God-object• Constraint surfaces:– Surfaces impeding motion– GO is outside (orientation test) and in theextension of the surface.– The HIP is inside the surface.motionedge/vertexConcavity: 2/3constraintplanes in 3Dmotion3DOF Haptics: God-object• Constraint plane equations:Ai⋅ x + Bi⋅ y + Ci⋅ z + Di= 0 3 planes at most• Energy function that will account for the distance.( )122E = ⋅ k ⋅ ( x − x ) ( ) ( )2HIP + y − y HIP + z − zHIP2• Define cost function using Lagrange multipliers.1222C = ⋅ k ⋅ ( x − x HIP ) + ( y − y HIP ) + ( z − zHIP) ) +23+ ∑ λi⋅ ( Ai⋅ x + Bi⋅ y + Ci⋅ z + Di)i = 1• Minimize, solving for x, y, z, λ 1, λ 2and λ 3.3DOF Haptics: God-object• Partial derivatives on x, y, z, λ 1 , λ 2 and λ 3yield 6 linear equations:⎛ 1 0 0 A1A2⎜⎜ 0 1 0 B1B2⎜ 0 0 1 C1C2⎜⎜ A1B1C10 0⎜⎜A2B2C20 0⎝ A3B3C30 0A3⎞ ⎛ x ⎞ ⎛ x HIP ⎞⎟ ⎜ ⎟ ⎜ ⎟B3⎟ ⎜ y ⎟ ⎜ y HIP ⎟C ⎟ ⎜ ⎟ ⎜ ⎟3 z zHIP⎟ ⋅ ⎜ ⎟ = ⎜ ⎟0 ⎟ ⎜ λ1⎟ ⎜ D1⎟0⎟ ⎜ ⎟ ⎜ ⎟⎟ ⎜λ2⎟ ⎜D2⎟0⎠ ⎝λ3⎠ ⎝ D3⎠• In case of less than 3 planes, theproblem has a lower dimension.3DOF Haptics: Virtual proxy• Ruspini et al., 1997.• Based on god-object.• Virtual proxy is a small sphere, instead of a point.Use configuration-space obstacles (C-obstacles),from robotics.• More formal definition of constraint planes.• Implementation of additional features, based onrelocation of the virtual proxy.3DOF Haptics: Virtual proxy• C-obstacles: for a spherical object, is reduced tocomputing offset surfaces at a distance equal tothe radius of the sphere.• Check the HIP against the offset surface.• This is done to avoid problems with small gaps inthe mesh.B13

3DOF Haptics: Virtual proxy3DOF Haptics: Virtual proxy• Finding the virtual proxy is based on an iterative search.• Basically, find subgoals based on the same distanceminimization as for the god-object.• At each subgoal, all the planes that go through that pointare potential constraints. The minimum set of activeconstraints is selected.HIP(t)Perform collisiondetection betweenthe path of the HIPand the C-obstacles• If the subgoal is in free space, set as new subgoal theHIP. The path might intersect the C-obstacles. Add thefirst plane intersected as a constraint and the intersectionpoint as the current subgoal.HIP(t+∆t)• The process ends when the virtual proxy becomes stable.3DOF Haptics: Virtual proxy3DOF Haptics: Virtual proxyHIP(t)Set the subgoal and theconstraint plane(s)HIP(t)Find a new subgoalusing the active planesand the minimizationbased on LagrangemultipliersHIP(t+∆t)HIP(t+∆t)3DOF Haptics: Virtual proxy3DOF Haptics: Virtual proxyHIP(t+∆t)HIP(t)Since the subgoal is infree space, drop theconstraints, set the HIP asthe new subgoal andperform collisiondetection between thepath and the C-obstaclesHIP(t+∆t)HIP(t)Recompute subgoalwith new constraintsB14

3DOF Haptics: Virtual proxy3DOF Haptics: Virtual proxyHIP(t+∆t)HIP(t)The path to the newsubgoal intersectsanother plane, sothis is added to theset of constraintsHIP(t+∆t)HIP(t)Compute activeconstraints (in 2Dthere are only 2) andfind subgoalFor this example,this is the finalposition of thevirtual proxy3DOF Haptics: Virtual proxy• Quadratic programming approach:– The constraint planes define an open convex region(bounded by the plane at infinity).– The function to minimize is the distance from the hapticdevice (HIP) to the new subgoal (VP i+1):C = VPi + 1 − HIP(t + ∆t)Quadratic function– The translation from the current location to the newsubgoal cannot intersect the constraint planes. Definelinear constraints based on the normals of the planes.N( VP −VP) 0i ⋅ i + 1 i ≥Linear constraints3DOF Haptics: Virtual proxy• Special case: simplified approach.– Let’s look at the convex region defined by theconstraint planesPlane at ∞VertexPlane at ∞Plane at ∞Plane at ∞EdgeFace (all surroundedby the plane at ∞)3DOF Haptics: Virtual proxy3DOF Haptics: Virtual proxy– Normals (n) become points (-n) in a dual space (Gaussmap).– The plane at infinity is the origin (o).– Points (-n) are joint together if the associated planesintersect.– The position of the virtual proxy also has a dual:HIP −VPiP =HIP −VPi– The vertices of the closest feature to P are the duals ofthe active constraint planes that are used for theminimization with Lagrange multipliers.B15

3DOF Haptics: Virtual proxy• Force output: PD (proportional- derivative) control. Producesa force that will try to keep the VP and the HIP at the sameposition.VPF* FSimulationPD control ActuatorsHIPUser*d( VP − HIP)F = Kp⋅(VP − HIP)+ Kd⋅dtIt’s like a spring+damper, butthe authors look at it from acontrol engineering approach3DOF Haptics: Additionalfeatures• Force shading by Basdogan et al., 1997.– Interpolate per-vertex normals using baricentriccoordinatesv 3Ai⋅ NiA Ai = 1N'2 1 N'=N =3N'AiA 3∑v2i = 1v 1– Effect: edges and vertices seem ‘rounded’3∑3DOF Haptics:Additional Features• Force shading by Ruspini et al., 1997.– Modify the position of the VP.– A subgoal (HIP’) is computed changing thenormal of the plane.– This subgoal replaces the HIP, and the finalVP is computed.Force-shadedOriginalplaneplaneHIPVPHIP’3DOF Haptics:Additional Features• Other effects by Ruspini et al., 1997.– Friction.– Textures.– Deformation.• All of them are achieved performingoperations that modify the position of the VP.3DOF Haptics:Additional Features• Friction by Hayward and Armstrong, 2000.– Model friction as an elasticity between the actualcontact point (x) and a fictitious point, called adhesionpoint (w).– The adhesion point remains static or follows theactual contact depending on the ‘state’ (sliding,w xstatic…)3DOF Haptics: H-Collide• What about the collision detection?– Spatial uniform decomposition of space. Locate the pathtraversed by the probe in that grid, using a hash table.– Test the path against an OBBTree. Optimized test.– Use frame-to-frame coherence, caching the previouscontact, and perform incremental computation.– When the leaf intersects, compute the surface contact point(SCP). Pass this to GHOST (haptic rendering library usedby the Phantoms).B16

3DOF Haptics: H-CollideThe EndSpatial partitioning with hash tableRay Vs. OBBTree testSCP computationSCPFor more information, seehttp://gamma.cs.unc.edu/interactivehttp://gamma.cs.unc.edu/HCollideB17

Course: Recent Advances in Haptic Rendering and ApplicationsSession II: 6-DoF Haptic Rendering for Object-Object InteractionMotivationIntroduction to 6-DoFHaptic DisplayBill McNeelyBoeing Phantom Workshttp://www.boeing.com/phantombill.mcneely@boeing.com• Why six degrees of freedom?– Allowable motions of an extended rigid object– E.g., 3 Cartesian coordinates plus 3 Euler angles• Basic to most real-world tasks– Virtual prototyping– TrainingWhy is 6-DOF so difficult?• Detect all contacts of interacting objects– Assume rigid bodies, but otherwise no a prioriconstraints on shape or motion• Calculate forces and torques– For hard contact / contact avoidance• 1000+ Hz haptic update rate– 10+ Hz graphic update rateKey decisions• Collision Detection– Choice of representation and algorithm• Interaction Paradigm– Penalty forces– Virtual coupling– Newtonian dynamics / Quasi-static approximation– Single user vs. collaborationFurther decisions• Decouple haptic and simulation loops?– Use intermediate representations?• Force type and quality– How hard does hard contact feel?– How free does free-space feel?• Repulsive forces?• Force artifacts / stability considerationsThree Approaches• Voxmap PointShell (VPS)– W. McNeely, K. Puterbaugh, and J. Troy• Sensation-Preserving Simplification– M. Otaduy and M. Lin• Spatialized Normal-Cone Hierarchies– D. Johnson and E. CohenB18

Course: Recent Advances in Haptic Rendering and ApplicationsSession II: 6-DoF Haptic Rendering for Object-Object InteractionVPS decisionsVoxel Sampling forSix-DoF Haptic RenderingBill McNeelyBoeing Phantom Workshttp://www.boeing.com/phantombill.mcneely@boeing.com• Voxel representation; Point-voxel sampling– 3-level 512-tree voxel hierarchy• Voxels / chunks / hyperchunks• Penalty forces; Virtual coupling• Tactical degradation of force quality– Free-space viscosity artifact• No decoupling of haptic and simulation loopsVPS collision detectionVPS interaction paradigm• Voxel sampling• Penalty ForcesVirtual couplingSome myths about VPS• “Cannot simulate multiple moving objects”• “Not scalable to large scenarios”• “VPS videos betray a stability problem”• (Your favorite myth here)VPS progress since 1999• 40x improvement in spatial accuracy– Applied to the design of Boeing 777-300ER• Collaborative haptics• See “Advances in Voxel-Based 6-Dof HapticRendering”B19

Geometrical Awareness (2D)• Vertex-edge contacts determine dynamicallycorrect behavior of rigid polygons in 2D– “Edge” includes verticesGeometrical Awareness (3D)• Vertex-surface and edge-edge contactsdetermine dynamically correct behavior ofrigid polyhedra in 3D– “Surface” includes edges– “Edge” includes verticesVoxel Based Distance Fields• Voxels marked with distance-to-contact• 3 types of fields: Surface, edge, vertexTemporal Coherence• Anticipate force-generating collisions• Sample fewer points using dead reckoning– Sampling rate varies inversely with distance-tocontact• Slow down the motion if necessary– Prime directive: Avoid undetected contacts!– Accept a free-space viscosity artifact in tradeDistance-to-contact queuesMaxTravel• Mark each point with distance-to-contact• Order points into distance-specific queues– Points free to migrate between queues• (Partially) traverse each queue every frame• Extend this scheme to the voxel hierarchy– Hierarchical temporal coherenceDefinition: Maximum allowable travel of any point during current frameMaxTravel = (nCapacity – nMandatory) / Σ (n i / (0.5 voxelsize · i))where:nCapacity = number of point-voxel tests CPU can perform in 1 msnMandatory = number of “mandatory” (contacting) pointsi = index of “discretionary” (non-contacting) queuen i = number of points currently occupying queue iNumber of discretionary tests = MaxTravel · n i / (0.5 voxelsize · i)B20

What happens whennMandatory exceeds nCapacity?• Graceful degradation– If nMandatory exceeds (0.95 · nCapacity), thentest every mandatory point plus (0.05 · nCapacity)discretionary points– Allow frame rate to fall below 1000 Hz• Hopefully the haptic interface tolerates this situation• Add more processing power?– Or use a larger voxel sizeScalability characteristics• CPU cycles per frame ~ O( 1/voxelsize 2 )• Amount of voxel data ~ O( 1/voxelsize 3 )– O( 1/voxelsize 2 ) at currently realistic voxel sizes• Multiple moving objects– CPU cycles ~ O( nMovingObjects 2 )Large-scale VPS haptics VPS-enabled FlyThru ®• Client-server architecture• 777-300ER main landing gear at ~1mm accuracy– 40,476 triangles moving (1.1 M points)– 2.7M triangles static (1.8 G voxels)• Dynamic pre-fetching of voxel data– SensAble Technologies PHANTOM ® 1.5/6DOF– Bi-Xeon 2.8 GHz with 2 GB RAMEngine exampleVPS-enabledCATIA®/Haption® prototypeInteractive assembly simulation with force-feedbackinside CATIA V5EuroHaptics 2004, Munich, Germany, June 5-7, 2004jerome.perret@haption.comHaptic device: Virtuose 6D35-45 by HaptionSoftware solution: Catia V5 R11 using VPS/PBMCAD model: Courtesy of RenaultB21

CATIA/Haption demoCollaborative 6-DOF haptics• 1.5mm voxel size• 36,565 trianglesstatic; 2,354moving• Voxelization time1.5 minutes• Bi-Xeon 2.4 GHzwith 1 GB RAM• Maximum force35N, torque 3NmCollaborative 6-DOF HapticsVirtual Swordplay• “Type 2” cross-user virtual coupling– Ensures stability regardless of latencyNon-planar manifold contactWhy is it a difficult scenario?• Geometrical awareness loses effectiveness• Extreme non-convexity is guaranteed• Why not use formal kinematic constraints?– Tricky transitions between unilateral and quasipermanentconstraints– System complexity increasesB22

VPS experimental approach• Allow slight surface interpenetration• Retract pointshell slightly– E.g., by ½ voxel along inpointing surface normal• Ball and socket example– At voxel size of (0.01 · diameter), MaxTravel allowsmaximum rotational speed of 0.5 revolution/secB23

Course: Recent Advances in Haptic Rendering and ApplicationsSession II: 6-DoF Haptic Rendering for Object-Object InteractionOutlineSensation Preserving Simplificationfor 6-DoF Haptic DisplayMiguel A. OtaduyETH-Zurichhttp://graphics.ethz.ch/~otmiguelotaduy@inf.ethz.ch• Motivation• Sensation Preserving Simplification• Multiresolution Collision Detection• Rendering Pipeline• ConclusionBottleneck: Collision DetectionPerceptual Motivation• Input:– Object A: m triangles Object B: n triangles• Query:– Return all pairs of triangles closer than d• Worst case scenario: O(mn)– Parallel close proximity (large areas closer than d)• Desired force update rate ~1kHz• Larger contact area Lower resolution• Supported by studies on feature detection[Klatzky and Lederman 1995, Okamura and Cutkosky 1999]Goal: Adaptive ResolutionGoal: Adaptive Resolutionlow reshigh resB24

Levels of DetailBounding Volume Hierarchies• Widely used in graphics [Hoppe 1996,Garland and Heckbert 1997, Lindstrom and Turk 1998…]• Adaptive, view dependent [Luebke 1997,Hoppe 1997], but based on visual error metrics• Hierarchical data structures for fast pruningof non-colliding geometry• Many different bounding volumes (AABBs,OBBs, spheres, k-dops, convex hulls…)ApproachContact Levels of Detail (CLODs)• Compute contact information usingmultiresolution representations (LODs)• Select LODs independently for each contact,based on local information• Problem: Integration of LODs with BVHs forfast collision detection• Unique hierarchy with dual functionality:LODs and BVH– Exploit hierarchical nature of both LODs and BVHs– Create LOD hierarchy and BVH simultaneously– Descend on BVH = Refine LODsOutline• Motivation• Sensation Preserving Simplification• Multiresolution Collision Detection• Rendering Pipeline• ConclusionCreation of CLODs• Initial meshM 0B25

Creation of CLODs• Initialize convex decompositionCreation of CLODs• Simplification + Filtering stepsM 0M 0 M 1c 0,0c 0,0c 0,1c 0,1c 0,jconvex pieces {c 0,0 ,… c 0,j } BVsc 0,jconvex pieces {c 0,0 ,… c 0,j } BVsCreation of CLODs• Maintain convex decompositionCreation of CLODs• Filtering Convexification. Merge BVsM 0c 0,0c 0,1M 1c 0,0c 0,1c 0,jc 0,jM 0c 0,0c 1,0c 0,1c 1,kc 0,jM 1merging convex piecesCreation of CLODs2D Example• Bottom-up constructionM 0c 0,0c 1,0c 0,1c 1,kc 0,jM 1M iM nc i,0c i,lc n,0Initial meshB26

2D Example2D ExampleConvex surface decompositionBVH construction:convex pieces2D Example2D ExampleSimplificationFiltered edge collapse(to enforce convexity constraints)Filtered Edge Collapse• Goals– Surface decimation– Filter detail– Convexification• Constraints– Convexity constraints– Preserve topologyFiltered Edge Collapse1. Edge collapse initialization. Quadric errormetrics [Garland and Heckbert 1997]2. Unconstrained filtering [Guskov et al. 1999]3. Optimization problem: minimize distance tounconstrained position subject to localconvexity constraints4. Bisection search: preserve topology andglobal convexity constraintsB27

2D Example2D ExampleFiltered edge collapse(to enforce convexity constraints)BVH construction:merge convex pieces2D Example2D ExampleSimplificationSimplification2D Example2D ExampleBVH constructionSimplificationB28

2D Example2D ExampleSimplificationSimplification2D Example2D ExampleSimplificationBVH constructionExample of HierarchyExample of HierarchyInput obj: 40K trisLOD 0: 11323 pieces LOD 3: 1414 piecesInput obj: 40K trisLOD 0: 11323 pieces LOD 1: 5661 piecesLOD 6: 176 piecesLOD 11: 8 piecesLOD 14: 1 pieceLOD 2: 2830 piecesLOD 4: 707 piecesLOD 7: 88 piecesB29

OutlineCollision Detection: BVHs• Motivation• Sensation Preserving Simplification• Multiresolution Collision Detection• Rendering Pipeline• Conclusionaba 1 a 2b 1 b 2BVHs of objectsA and BCollision Detection: BVHsCollision Detection: BVHsaba 1 a 2 b 1 b 2aba 1 a 2b 1 b 2Bounding VolumeTest Tree (BVTT)BVHs of objectsA and BBounding VolumeTest Tree (BVTT)BVHs of objectsA and BCollision Detection: BVHsCollision Detection: BVHsno collisioncollisionaba 1 a 2b 1 b 2no collisioncollisionaba 1 a 2b 1 b 2Bounding VolumeTest Tree (BVTT)BVHs of objectsA and BBounding VolumeTest Tree (BVTT)BVHs of objectsA and BB30

Collision Detection: BVHsCollision Detection: BVHsno collisioncollisionaba 1 a 2b 1 b 2no collisioncollisionaba 1 a 2b 1 b 2Bounding VolumeTest Tree (BVTT)BVHs of objectsA and BBounding VolumeTest Tree (BVTT)BVHs of objectsA and BCollision Detection: BVHsCollision Detection: BVHsno collisioncollisionaba 1 a 2b 1 b 2no collisioncollisionaba 1 a 2b 1 b 2contactBounding VolumeTest Tree (BVTT)BVHs of objectsA and BCost proportional tosize of frontBVHs of objectsA and BCollision Detection: CLODsCollision Detection: CLODs• Sensation-preserving selective refinement– Exploit multiresolution representations– Descend BVH = Refine LOD– Return contact information at appropriateresolution– Reduce cost of collision detection drasticallyno collisioncollisionSelective Refinementaba 1 a 2 b 1 b 2BVHs of objectsA and BB31

Collision Detection: CLODsCollision Detection: CLODsno collisioncollisionno refinementaba 1 a 2b 1 b 2no collisioncollisionno refinementaba 1 a 2b 1 b 2Selective RefinementBVHs of objectsA and BRaise the FrontBVHs of objectsA and BCollision Detection: CLODsError Metricsno collisionno refinementcollisionr ir jAdaptive resolution (r)aba 1 a 2b 1 b 2BVHs of objectsA and B• Refine if error(a, b) = s*ab > ε• Error: weighted surface deviations*ab⎛ sasmax⎜,2⎝ rarb=Db2⎞⎟⎠s: surface deviationr: resolutionD: estimated contact area• ε: 3% of object radius (user studies)BenchmarksForce Profile: Jawsupper jaw: 47,339 trislower jaw: 40,180 trisDual Pentium4 2.4GHzclub: 104,888 trisball: 177,876 trisupper jaw: 47,339 trislower jaw: 40,180 trisjoint: 137060 tris•Offline•Full res. models•Exact collisiondetection (Swift++)B32

Force Profile: JawsTimings: Golfupper jaw: 47,339 trislower jaw: 40,180 trisDual Pentium4 2.4GHz1000CONTACT QUERY (ms)5002003%0.3%0.03%exactclub: 104,888 trisball: 177,876 trisDual Pentium4 2.4GHz10050• Interactive• ε = 2.5%20• > 200ms10•Offline5•Full resolution•Exact collision2detection10 50 100frames150 200Timings: GolfTimings: Golf1000CONTACT QUERY (ms)5002003%0.3%0.03%exactclub: 104,888 trisball: 177,876 trisDual Pentium4 2.4GHz1000CONTACT QUERY (ms)5002003%0.3%0.03%exactclub: 104,888 trisball: 177,876 trisDual Pentium4 2.4GHz1001005050> 100x speed-up201052• ~2ms• Interactive• ε = 3%20105210 50 100frames150 20010 50 100frames150 200OutlineDecoupled Modules• Motivation• Sensation Preserving Simplification• Multiresolution Collision DetectionHaptic Deviceand Controllerforce &torqueGrasped ObjectSimulationforce &torqueCollision Detectionand Response• Rendering Pipeline• ConclusionB33

Virtual CouplingLinearized Contact Modelstiffness k cstiffness kHaptic Deviceand ControllerGrasped ObjectSimulationCollision Detectionand ResponseHaptic Deviceand ControllerGrasped ObjectSimulationCollision Detectionand ResponseVirtualCouplingVirtualCouplingLinearizedContactModelDecouple object simulation fromsynthesis of force feedbackDecouple collision detection from simulationGrasped Object SimulationMultires. Collision DetectionHaptic Deviceand ControllerImplicit Integrationof Rigid BodyDynamic SimulationCollision Detectionand ResponseHaptic Deviceand ControllerImplicit Integrationof Rigid BodyDynamic SimulationContact Levelsof DetailVirtualCouplingLinearizedContactModelVirtualCouplingLinearizedContactModelLarger range of stable stiffness valuesFast collision detection between complex objectsMultirate PipelineStability and ResponsivenessHaptic Deviceand ControllerImplicit Integrationof Rigid BodyDynamic SimulationContact Levelsof DetailVirtualCouplingHaptic Thread (1kHz)LinearizedContactModelContact ThreadB34

Outline• Motivation• Sensation Preserving Simplification• Multiresolution Collision Detection• Rendering Pipeline• ConclusionSummary• Contact levels of detail:– Multiresolution collision detection– Unifies LODs and BVHs– Per-contact adaptive selection of resolution– Error metrics based on perceptual studiesMain Results• Stable and responsive 6-DoF hapticrendering of complex models– Models: 40K – 170K (in complex contactscenarios)– Collision detection update rates: 300Hz – 500Hz– Collision detection error: 2.5% - 3% of objectbounding radii– Speed-up: up to 100xGeneralization• CLODs data structure independent of BV• Application to rigid body simulation• Extension of sensation preservingsimplification to other frameworks:– Voxel sampling [McNeely et al. 1999]– Normal cone hierarchies [Johnson and Cohen 2001]Limitations• Lack of containment virtual prototypingapplications do not allow interpenetration• Limited simplification aggressiveness due totopological constraints• Static LODs and ‘popping’ effectsLimitations• Driving observation: small features cannot beperceived if the contact area is large• Does not hold for:– Highly correlated features– Tangential motion• Solution: ‘Haptic Rendering of TexturedSurfaces’ (later in this course)B35

ReferencesSensation Preserving Simplification for Haptic Rendering.Miguel A. Otaduy and Ming C. Lin.In Proc. of ACM SIGGRAPH 2003.http://gamma.cs.unc.edu/LODHaptics/Stable and Responsive Six-Degree-of-Freedom HapticManipulation Using Implicit Integration.Miguel A. Otaduy and Ming C. Lin.In Proc. of World Haptics Conference 2005.http://gamma.cs.unc.edu/SR6DoF/B36

Course: Recent Advances in Haptic Rendering and ApplicationsSession II: 6-DoF Haptic Rendering for Object-Object InteractionHaptic Rendering of Polygonal ModelsUsing Local Minimum DistancesDavid Johnson*University of Utahhttp://www.cs.utah.edu/~dejohnsodejohnso@cs.utah.eduMotivation• Virtual Prototyping– Replace expensive and hardto make physical prototypes– Check assembility andaccessibility– Test human ergonomics andaesthetics*Joint work with Elaine Cohen and Pete WillemsenHaptic Computation• Typically compute penetration depthHaptic Computation• Typically compute penetration depthHaptic Computation• Typically compute penetration depth• Compute restoring forceHaptic Computation• Typically compute penetration depth• Maintain restoring forceB37

Haptic Computation• Typically compute penetration depth• Maintain restoring forceHaptic Computation• Typically compute penetration depth• Maintain restoring forceHaptic Computation• Typically compute penetration depth• Maintain restoring forceHaptic Computation• Typically compute penetration depth• Maintain restoring forceDifficulty Using CollisionStatus• Waiting until interpenetration violates realworldconstraints• In virtual prototyping application, we want tocheck whether models can fit – penetratingmodels reduces validity• Penetration depth difficultLocal Minimum DistanceApproach• Find local minimumdistances betweenpolygonal models• Convert distances intoforces (translational,torque)• Forces guide objects toprevent interpenetrationB38

ExampleWhy are Local ClosestPoints Adequate?• One global distance notenough• All pairs within a cutoffdistance too many• Collisions are representedby pairs of contact points• Move objects apart andpairs of points becomelocal closest pointsLocal minimum distance• CAGD approach:22D ( u,v,s,t)= F(u,v)− G(s,t)Extrema when( F − G)⋅ F =u 0( F − G) ⋅ F = 0v( F − G)⋅ G =s 0( F − G) ⋅ G = 0.tComputing LMD’s betweenPolygonal Models• Build bounding hierarchy• Prune hierarchy based on normals, notdistance• Leaf tests between triangles• Normals are the key to describing localminimum distanceSpatialized Normal ConeHierarchyConstructing the hierarchySemiangle θaxisradiuscenterI3D 2001 paper• Use PQP package (UNC) to constructspatial bounding volume hierarchy• At each node– find average normal and max spread tomake normal coneB39

Pruning the HierarchyPruning the Hierarchy• Create double conebetween boundingspheres( F − G)⋅Fu= 0( F − G) ⋅Fv= 0( F − G)⋅Gs= 0( F −⋅Gt= 0• Check normal cone ofone model againstdouble cone( F − G)⋅Fu= 0( F − G) ⋅Fv= 0( F − G)⋅Gs= 0( F −⋅Gt= 0Pruning the HierarchyPrune with Cutoff Distance• Check normal cone ofother model againstdouble cone( F − G)⋅Fu= 0( F − G) ⋅Fv= 0( F − G)⋅Gs= 0( F −⋅Gt= 0• We don’t want allLMDs, just thosenearby• Modify computation toreject nodes that arefurther than cutoff• This quickly prunesmany nodesCutoff ExampleResults• Can handle arbitrarymodels• Explore concaveregions• Provide sensations ofcontact and torque toguide modelB40

Example: Concave regionsExample: Non-mechanicalModelAcceleration with LocalDescent• First, do a global searchAcceleration with LocalDescent• Then, as the object moves, do local updatesAcceleration with LocalDescent• Then, as the object moves, do local updatesAcceleration with LocalDescent• Then, as the object moves, do local updatesB41

Acceleration with LocalDescent• Concurrently, repeat global searchAcceleration with LocalDescent• Merge in new LMDsApplication Design• Multi-threaded design– Global search– Local update and force computation– Graphics• Communicate through shared datastructures• Dual 2.4GHz HT P4, shared memory,Linux/GNU computerLocal Update• Check neighbor triangles tomove to a new local minimum• Check triangles in one model’sneighborhood against triangles inother model’s neighborhood.• Do this for each local minimumdistance pair• Can compute ~1,000,000triangle distance pairs persecondLocal Update• Move and find neighborhoodLocal Update• Compute closest points between trianglesB42

Local Update• Find new minimum and neighborhoodLocal Update• Find new distancesLocal Update• Find new neighborhood and distancesLocal Update• Find new neighborhood and distancesLocal Update• Local search has convergedResults• Models with10k-100ktrianglesB43

More ResultsComparison – Just Global• Haptic system ismuch smoother andmore stable withlocal updateComparison – Local UpdateAccessibility Application• Save position and orientation at each time step• If collision occurs– Revert to safe state several time steps before collision– Continue simulation• Visualize path at end of trial– Sample saved positions– Draw model at each sample– Allow manipulation of scene to verify pathAccessibility ApplicationConclusion• High-performance haptic rendering– Using local techniques provides multiple orders ofmagnitude speedup• General polygonal environment• Accurate results– Adjustable clearance distances• Keeps the human in the loop, but hapticsprovides guidanceB44

Course: Recent Advances in Haptic Rendering and ApplicationsSession III: Haptic Rendering of Higher-Order PrimitivesWhy Sculptured Models?Haptic Rendering ofSculptured ModelsElaine Cohen*University of Utahhttp://www.cs.utah.edu/~cohencohen@cs.utah.edu• Prevalent format formanufacture andanimation• Compact and exactrepresentation• Surface normals varysmoothly* Joint work with Tom Thompson, Don Nelson, and David JohnsonBasic Spline BackgroundBasic Spline Background• Parametric function– Simple example:interpolating two pointstP + ( − t P0 1 )1• Parametric function– Simple example:interpolating two pointstP + ( − t P0 1 )1P1– Weighting control pointsP1P0P0Basic Spline BackgroundBasic Spline Background• Parametric function– Simple example:interpolating two points– Weighting control points– Basis functionstP + ( − t P0 1 )P11• Extends to– higher-degree basisfunctions– piecewise functions– Higher dimensionaldomainsP0B45

Spline Background - Trims• A trim is a curve in thedomain– Often piecewise linear• Removes part of domain andthe corresponding surface• Glues together multiplesurfaces– Result of Boolean operationsDistance Between a Pointand Surface• Squared distance betweenpoint and surface2D ( u,v)= ( S(u,v)− P) ⋅( S(u,v)− P)• Extrema at simultaneouszeros of partials( S(u,v)− P)⋅Su⎤( − ) ⋅⎥ ⎦⎡F = ⎢⎣ S(u,v)PSvFinding Simultaneous Zeros• Multidimensional Newton’s MethodJ ⋅ ∆p = −F• For minimum distance, this becomes⎡(S− P) ⋅S⎢⎣(S− P) ⋅Suuuv+ S+ Suu⋅ S⋅Suv(S − P) ⋅S(S − P) ⋅Suvvv+ S+ Suv⋅ S v ⎤⎡∆u⎤⎡(S− P) ⋅ Su⎤⎥⎢⎥ = −⋅⎢ ⎥S v ⎦⎣∆v⎦⎣(S− P) ⋅Sv ⎦Make Linear Approximation• Increase speed– SARCOS arm onembedded system• Good enough with highcoherenceDirect Parametric Tracking(DPT)Initial State Movement Project Computeparametricchange and newsurface positionTrimmed Models• Trims add sharp edges in model• Need to be able to transitionfrom one surface to neighbor• Need fast inside-outside test fortrim loopB46

Transitioning• State change while tracing• Three forms– Across surface boundary– Along surface intersection– Off of the modelEdge Tracing and Release• Slide along trim in 3D to locally close point– Project probe onto current segment– If past endpoint slide– Continue in one direction until minimum reached• Check for release onto surface– Evaluate surface and apply DPT– If point doesn’t release try adjacent surface– If point still doesn’t release make special normalTrim Intersection• Discrete movement onsurface• Brute force approach notfeasible• Grid approach– Boundary on bounding box– Locality– Walk algorithmGrid Data• Goblet– 3 surfaces– 254 segs– 13 max– 3 mean– 89% empty• Gear• Crank Shaft• 22 surfaces • 73 surfaces• 1256 segs • 412 segs• 15 max• 36 max• 4 mean• 4 mean• 92% empty • 89% emptyExampleExample2B47

Model-model rendering• Maintain penetration between two models• Use Newton’s method to initialize, thenintegrate to maintainDifferential SurfaceKinematics• Using relations from differential geometryand robotics calibration, the general resultfor the relation between parametric contactcoordinate velocity and body velocity isderived:Surface-to-SurfaceTracingJust integrate:Surface-to-SurfaceManagementThree step tracking processProvides closed form local collisiondetection updates.Surface ContactsExample• Possible multiplecontacts– Initializing method musttrack potential contacts• Newton’s methodinitializes closest points• Near contact, stableintegration takes over• Maximum penetrationis maintainedB48

Course: Recent Advances in Haptic Rendering and ApplicationsSession IV: Rendering of Textures and Deformable SurfacesWearable Vibrotactile DisplaysA Living Proof thatHaptics Works!TadomaHong Z. TanHaptic Interface Research LaboratoryPurdue Universityhttp://www.ece.purdue.edu/~hongtanhongtan@purdue.eduPhotograph by Hansi Durlach, 1980.Sensory Substitution:Text to VibrationOptaconOptohaptSensory Substitution:Image to VibrationTVSSPhotograph Courtesy ofTelesensory Corp.Geldard, F. A. (1966). Cutaneous coding of optical signals:The optohapt. Perception & Psychophysics, 1, 377-381.White, B. W., Saunders, F. A., Scadden, L., Bach-Y-Rita, P., & Collins, C. C.(1970). Seeing with the skin. Perception & Psychophysics, 7(1), 23-27.Sensory Substitution:Speech to Vibration“Felix”Tactaid VIIConsiderations for WearableVibrotactile Displays• Wearability• Body site• Intended User• Training• Sensory adaptation• Integration with visual and auditory displaysPhotograph byAlfred Eisenstaedt,1950.Weisenberger, J. M., & Percy, M. E. (1994). Useof the Tactaid II and Tactaid VII with children.The Volta Review, 96(5), 41-57.• What is the most effective way tocommunicate via a vibrotactile display?B49

Haptic Cueing of VisualSpatial AttentionCollaborators: Rob Gray, Jay YoungApplicationsBlankFrame 2BlankFrame 1H. Z. Tan, R. Gray, J. J. Young, and R. Traylor, “A haptic back display for attentional anddirectional cueing,” Haptics-e: The Electronic Journal of Haptics Research, 3(1), 2003.Situation AwarenessMilitary Silent OperationRupert, A. H. (2000, March/April). An instrument solution for reducing spatialdisorientation mishaps - A more "natural" approach to maintaining spatial orientation.IEEE Engineering in Medicine and Biology Magazine, 19, 71-80.Veen, H.-J. v., Spape, M., & Erp, J. v. (2004). Waypoint navigation on land: different waysof coding distance to the next waypoint. Proceedings of EuroHaptics 2004, 160-165.Jones, L. A., Nakamura, M., & Lockyer, B. (2004). Developmentof a tactile vest. Proceedings of HAPTICS '04, 82- 89.Limitations in TactileInformation ProcessingTactile Change DetectionNumerosity Judgments7Offset6Resp onses54321Onset01 2 3 4 5 6 7Number of stimulitimeB50

Conclusions• Haptics technology has reached a point wherewearable vibrotactile displays are possible• Many challenges remain in developing lightweightand robust actuators• We’ve yet to develop a tactile vocabulary thatmatch the sensory capability of the body surface• More basic research is needed to explore theinformation-processing limitations in using wearablevibrotactile displaysB51

Course: Recent Advances in Haptic Rendering and ApplicationsSession IV: Rendering of Textures and Deformable SurfacesOutlineTowards Realistic HapticRendering of Surface TextureSeungmoon Choi and Hong TanHaptic Interface Research LaboratoryPurdue Universityhttp://www.ece.purdue.edu/~hongtanhongtan@purdue.edu• Introduction– Texture– Perceived Instability• Psychophysical Evaluations• Sources of Perceived Instability• ConclusionsWhat is Texture?Haptic Texture Rendering• What is texture?– Texture = micro geometry of a surface– Shape = macro geometry of a surface• Why study haptic texture rendering?– To enrich the sensory attributes of virtual objects(L = 2.5 mm A = 50 µm)– To provide well-controlled stimuli for psychophysicalstudies• How to render texture haptically?– Haptic texture information can be effectively conveyedthrough temporal cues alone (e.g., Katz’ observations)– Point-contact probe-mediated force feedback devicescan adequately render virtual textures• Past work has focused on the developments of fastcomputational algorithms– Massie (1996) − force magnitude– Ho et al. (1999) − force magnitude & directionPerceived InstabilityOutline• Definition: Any unrealistic sensation that can not beattributed to the physical properties of simulatedtextured surfaces (e.g., buzzing).• Types of Perceived Instability– “Buzzing”– “Aliveness”– “Ridge Instability”• Can be caused by both device instability andinadequate environment model• Introduction– About Texture– Perceived Instability• Psychophysical Experiments• Sources of Perceived Instability• ConclusionsB52

ApparatusTexture Model• PHANToM 1.0A (SensAble Technologies, MA)• Free Exploration vs. StrokingMain Findings• Largest stiffness values for “clean” texture:0.01 to 0.78 N/mm (“soft corduroy”)• Largest stiffness value for “clean” flat wall:1.005 N/mm• We need a better understanding of the types ofperceived instabilities and their underlying causes• The goal is to be able to render “harder” textureswithout any perceived instabilityThree Types ofPerceived Instabilities• Buzzing– Perceived as high-frequency noises (“vibrating” textures)• Aliveness– Perceived as low-frequency force variations (“pulsating”textures) when the stylus is stationary• Ridge Instability– When resting on the peak of a sinusoidal ridge, the stylusis being pushed into the valley.OutlineBuzzing• Introduction– About Texture– Perceived Instability• Psychophysical Experiments• Sources of Perceived Instability• Conclusions|P z (f)|: dB re 1µm peak, Perceived Magnitude: dB SLB53

What Causes Buzzing?AlivenessPHANToM Frequency ResponseMeasured at the origin of the PHANToMwith the stylus pointing to the +z directionSignal duration = 400 msWhat Causes Aliveness?Passivity Observer• Our hypothesis was that Aliveness camefrom the texture rendering algorithm, but notan unstable haptic interface• To test our hypothesis, we implemented aPassivity Observer (Hannaford & Ryu, 2002)on measured force and position dataPassive Rendering withAlivenessWhat Causes RidgeInstability?p x (t): mm, p z (t): mm, F zW (t): N, PO (t): Nmm/secB54

Conclusions• Many virtual textures contain perceived instability• Stroking is the preferred method for textureperception because it has a larger stiffnessthreshold for perceived instability• Perceived instability can occur even when thehaptic interface system is passive• Research based on both control and perceptionare necessary in order to achieve perceptuallyrealistic texture renderingB55

Course: Recent Advances in Haptic Rendering and ApplicationsSession IV: Rendering of Textures and Deformable SurfacesOutlineHaptic Rendering ofTextured SurfacesMiguel A. OtaduyETH-Zurichhttp://graphics.ethz.ch/~otmiguelotaduy@inf.ethz.ch• Motivation• Algorithm Overview• Synthesis of the Force Model• Penetration Depth on the GPU• Experiments and Results• ConclusionIntroductionIntroduction• Geometric surface texture:– Compelling cue to object identity– Strongly influences forces during manipulation• Objects with rich surface texture informationcannot be handled by state-of-the-art hapticrendering methods.ModelsOutlineCoarse geometricrepresentationsHaptictextures• Motivation• Algorithm Overview• Synthesis of the Force Model• Penetration Depth on the GPU• Experiments and Results• ConclusionB56

3-DoF Texture Rendering3-DoF Texture Rendering• 1 contact point on a textured surface– Minsky [1995], Ho et al. [1999]: high frequencyforces based on gradient of height fieldsimplifiedsurfacecontact pointheight field intexture map• 1 contact point on a textured surface– Siira and Pai [1996]: stochastic model– Pai et al. [2001]: auto-regressive model forroughness and friction6-DoF Texture Rendering• Object-object interaction– Contact cannot be described as point-surfacecontact– Force and torque output has 6-DoF; point contactonly has 3-DoFRendering Algorithm1) Compute contact information betweenlow-res models• A different rendering algorithm is requiredRendering Algorithm1) Compute contact information betweenlow-res models2) Refine contact information using detailgeometry stored in texturesRendering Algorithm1) Compute contact information betweenlow-res models2) Refine contact information using detailgeometry stored in textures3) Compute contact forces based on noveltexture force modelB57

Force Model Overview• Accounts for important factors identified byperceptual studies• Based on the gradient of inter-objectpenetration depth• GPU-based computation of directionalpenetration depthOutline• Motivation• Algorithm Overview• Synthesis of the Force Model• Penetration Depth on the GPU• Experiments and Results• ConclusionRelated Work:Perception & PsychophysicsRoughness Vs. Texture Spacing[Klatzky and Lederman 1999-present]• Studies on perception of textures through arigid probe by Klatzky and Lederman [1999-present]– Analyze effects of probe diameter, applied forceand exploratory speed– Inspiration for our force modellog (roughness)Probe Diameter (D)Applied Force (F)Exploratory Speed (v)log (texture frequency)Effect of Probe Diameter (D)[Klatzky and Lederman 1999-present]log (roughness)D+- Strong influence ofgeometryEffect of Applied Force (F)[Klatzky and Lederman 1999-present]log (roughness)+-FRoughness growswith applied forcelog (texture frequency)log (texture frequency)B58

Effect of Exploratory Speed (v)[Klatzky and Lederman 1999-present]Offset Surfaceslog (roughness)v+-Dynamic effects alreadypresent in haptic simulationSpherical probe trajectory = offset surfaceUse offset surface as descriptor of vibrationArbitrary objects ???How can we generalize offset surfaces?log (texture frequency)Penetration Depth: Definitionδ= Minimum translational distance toseparate two intersecting objectsPenetration Depth: Definitionδ= Minimum translational distance toseparate two intersecting objectsδDirectional PD: Definitionδ n = Minimum translation along n to separatetwo intersecting objectsDirectional PD: Definitionδ n= Minimum translation along n to separatetwo intersecting objectsδ nnnB59

Offset Surface and PDOffset Surface and PDOffsetsurfaceOffsetsurfaceTexturedsurfaceTexturedsurfaceOffset Surface and PDhForce Model• Penetration depth:– Applicable to arbitrary object-object interaction– Also used in previous single-point rendering methodsδδ ≡δ = hpenetration depth• Penalty-based potential field:1 U = kδ22Force ModelEffect of Geometry• Determine penetration direction n• Force and Torque = Gradient of energy:( F F F T T T ) = −∇U= −kδ( ∇δ)uvnu⎛ ∂δn∂δn− kδ⎜n1⎝ ∂u∂vvn∂δn∂θun∂δn∂θvn=∂δ⎞n⎟∂θn⎠• Force and torque proportional to gradient ofpenetration depthF F F T T T = −k∇( ) δ ( δ )uvHigh amplitude texture High derivative of penetration depth Large force/torqueMethod validated by Minsky [1995]nuvnnnB60

Effect of Applied Force• Normal force:F• Other forces and torques:Fun= −kδ∂δn= −kδn∂uLarger normal force Larger roughness effectn∂δn= Fn∂uOutline• Motivation• Algorithm Overview• Synthesis of the Force Model• Penetration Depth on the GPU• Experiments and Results• ConclusionDirectional PD: Definitionδ n= Minimum translation along n to separatetwo intersecting objectsδ nDirectional PD of Height Fieldsδ = maxn( h − h )ABBnhA − h BAnPD with Texture ImagesPD with Texture ImagesHigh-resSurfaceHigh-resSurfaceLow-resSurfaceLow-resSurfacennB61

PD with Texture ImagesPD with Texture ImagesHigh-resSurfaceHigh-resSurfaceLow-resSurfaceLow-resSurfacennPD Computation AlgorithmLow-Resolution Models……+ Texture ImagesStep 1: Approximate PDB62

Step 1: Approximate PDStep 2: Refined PDPass 1: Render GeometryPass 1: Texture MappingPass 1: Sample SurfacesPass 1: Project to PD DirectionB63

Discrete Height FieldsPass 2: Subtract Height FieldsPass 2: Copy to Depth BufferBinary Search for Max = PD[Govindaraju et al. 2004]TestTestB64

Gradient of PD( F F F T T T ) = −∇U= −kδ( ∇δ)uvnu⎛ ∂δn− kδ⎜n⎝ ∂uvn∂δn∂v1∂δn∂θun∂δn∂θvn=∂δ⎞n⎟∂θn⎠Gradient of PD• Central differences∂δnδn≈∂u( u + ∆u, v,n,...) −δ( u − ∆u,v,n,...)2⋅∆u• Recompute PD at 2 new object configurationsnOutline• Motivation• Algorithm Overview• Synthesis of the Force Model• Penetration Depth on the GPU• Experiments and Results• ConclusionExperiments• Offline analysis of force model• Quality of texture effects• Performance tests.Offline ExperimentsEffect of Probe DiameterStudies by Klatzky and LedermanSimulation results50num hk hDδ nvMaximum Acceleration2010521mm2mm3mmb h10.8 1 2 4 6Texture Spacing (mm)B65

Effect of Applied ForceEffect of Exploratory SpeedStudies by Klatzky and LedermanSimulation resultsStudies by Klatzky and LedermanSimulation results5050Maximum Acceleration2010520.29N0.58N0.87N10.8 1 2 4 6Texture Spacing (mm)Maximum Acceleration201055.4mm/s114.6mm/s222.2mm/s50.8 1 2 4 6Texture Spacing (mm)Roughness under TranslationRoughness under Rotationzyx8070605040Position (mm.)xyz0.90.80.70.60.5Forces (N)F xF yF zn6Motion along n (in mm.)4200 1000 2000 3000 4000 5000 60003020100-100 500 1000 15000.40.30.20.10-0.10 500 1000 1500Simulation frames2 Rotation around n (in deg.)0-20 1000 2000 3000 4000 5000 6000Simulation framesComplex ObjectsTimings: File and CAD Part1210864Time (in msec.)Total Frame TimeTexture ForcesCollision DetectionFile full-res: 285KtrisFile low-res: 632 trisCAD full-res: 658KtrisCAD low-res: 720tris200 100 200 300 400 500Number of contacts5Dual Pentium4 2.4GHzNVidia FX595000 100 200 300 400 500Simulation framesB66

Outline• Motivation• Algorithm Overview• Synthesis of the Force Model• Penetration Depth on the GPU• Experiments and Results• ConclusionSummary• Haptic rendering algorithm usinglow-res models and texture images• Force model inspired by psychophysicsstudies• Image-space algorithm for PD computation(implemented on GPU)Results• 500Hz force update rate with relativelysimple models• 100Hz-200Hz force update rate with complexmodelsLimitations• Cannot handle surfaces that cannot bedescribed as height fields in the contact region• Possible sampling-related aliasing• Limited stability with high PD gradient• Haptic rendering of interaction betweencomplex textured modelsFuture WorkReferences• Higher frequency textures• Deformable textured surfaces• Analysis of human factorsHaptic Display of Interaction between Textured Models.Miguel A. Otaduy, Nitin Jain, Avneesh Sud and Ming C. Lin.In Proc. of IEEE Visualization Conference 2004.A Perceptually-Inspired Force Model for Haptic TextureRendering. Miguel A. Otaduy and Ming C. Lin.In Proc. of the Symposium on Applied Perception inGraphics and Visualization 2004.http://gamma.cs.unc.edu/HTextures/B67

ElasticityModeling Deformable Objectsfor HapticsDinesh K. Pai(mostly in collaboration with Doug James)Rutgers University• Dynamics defined at infinitesimal scale– force -> stress– displacement -> strain– Hooke’s law relates stress to strain– Newton -> Cauchy• Hyperbolic partial differential equations• PDE + boundary conditions= Boundary Value Problem (BVP)Outline• Basics of Elasticity• Linear Elastostatic Models• Green’s Functions• Capacitance Matrix Algorithms• Haptic Interaction Issues• ExtensionsLinear Elastostatic Models3⎛• Simple but stableG∑⎜k = 1 ⎝• Captures many “global”aspects of deformation• Sufficient to only deal withboundary discretization• Significant precomputation• But inaccurate for somematerials, large deformation22∂ u 1 ⎞i∂ uk++ = 021 2⎟ bi∂xk− ν ∂xk∂xi⎠n = 0.5Interactive DeformationNumerical Discretization[James + Pai 99]• Most problems must be solved numerically– Finite Differences– Finite Element Method (FEM)– Boundary Element Method (BEM)• FEM => internal discretization; easy tohandle anisotropy and inhomogeniety• BEM=> only boundary discretization; easy tohandle mixed boundary conditionsB681

Key Idea: Green’s FunctionsGreen’s functions U relateu, vertex displacements top, applied vertex tractionsCan be computed analytically ifmaterial distribution is knownRelated work:Cotin et al. 96 “Condensation”u = U pAstley & Hayward 98 “Norton equivalents” Green’s FunctionsJames and Pai 99 “Green’s functions” via BEM=jjp jjFast Solution to BVP withGreen’s Functions• Usually few specified BV’s are nonzero• If s (out of n) non-zero BVs, _O(ns) time to evaluate v = Ξ vExample: BoundaryElementsN u + b = 03GWeaken, ∑k = 1Integrate∫c u + p * u d Γ = u * p dΓΓH u = G p∫ΓDiscretize⎛ ∂ u⎜⎝ ∂PointLoadat j2i2xk21 ∂ uk+1−2ν∂x∂xConstant Elementsig ijki⎞⎟ + bi= 0⎠Boundary Condition TypeChanges• Problem:– if BC changes, have to recompute Ξ–Ξ large and dense• Idea: ExploitcoherenceGreen’s Functionsfor Discrete BVP (via BEM)Fast ElastostaticDeformation=H u = G p• BC change swaps a block column of A=SPECIFY BCRedYellowBV specifiedBV unknown= H u = G p==REARRANGEINVERT LHS_ _A v = - A v_ _ _v = - A -1 A v = Ξ vGreen’s Functionss= +TA = A + δAE0sAs v = - AsvB692

Notation• E is n x s submatrix of IdentityM E “Extracts” columns from matrix MNo cost• Change in matrixAs= A0+ (A0−A0)EET• videoδA (A A )E• = −s 0 0Changed columnsTA = A + δAE = +s0s[James, Pai SIGGRAPH 99]Sherman-Morrison-Woodbury• Idea: Exploit coherence between BVPsT• If As= A0+ δAsE = +A-1s-10-10sT -1 −1 T -1[ I+E A A ] E A= A - A δAδs-by-s capacitance matrix (small)0s0Haptic Interaction [James & Pai01]• Need to update contact force at much higherrate (1 KHz)– Idea: use faster local model at contact point– Related work: [Astley & Hayward 98], [Balanuik00], [Cavsolglu & Tendick 00]• Need to support “point contact” abstraction(e.g., for PHANToM)Capacitance MatrixAlgorithm [James + Pai 99,01]• Exploit coherence using SMW formula• When s nodes switch, extract and factorC = -E T ΞE = s-by-s capacitance matrix– Henceforth,_v (0) = [ Ξ(I-EE T ) - EE T ] vv = v (0) + (I+Ξ)EC -1 E T v (0)E “extracts” the s columnsO(s 3 ) or betterO(n s)Local Models from theCapacitance Matrix Algorithm• Region of Interest– If output is needed only at d nodes (withextraction matrix E D),ETDv = ETDv (0) + (E DT(I+Ξ)E)C -1 E T v (0)in O(d s + s 2 ) time• If these are the same s contacting nodes,simplifies dramatically! _E T v = -C -1 E T v⇒ an exact “local buffer” for hapticsB703

Point Contact abstraction• Real point contact produces infinitetractions (and therefore largerdisplacements on finer mesh)• Use vertex masks to distributedisplacement over finite area[James & Pai 01]DyRT [James & Pai 02]Dynamicphysically-based modal deformationResponseto bone-based animationTexturesprecomputed, sampled, and renderedalmost entirely on graphics hardwareCapacitance Matrix in HapticsDyRT movieMultiresolution Green’sFunctionsSurgical Simulation[James and Pai 02][James & Pai 02]B714

Resources• Gibson and Mirtich “A Survey of Deformable Models inComputer Graphics” MERL Tech Report TR-97-19, 1997– www.merl.com• James & Pai SIGGRAPH 99 contains a review of elasticmodels– www.cs.ubc.ca/~pai/papers/JamPai99.pdf• James & Pai “A Unified Treatment of Elastostatic and RigidContact Simulation for Real Time Haptics”, to appear(2001) in Haptics-e The Electronic Journal of HapticsResearch– www.haptics-e.orgExploiting reduced coordinatesfor collision detectionTraditional collision detectors don’tcare about reduced coordinatesqSimulator..:Collision DetectorBD-Tree [James&Pai SIGGRAPH 04]ExtensionsBounded Deformation treeIdea: Parameterize bounding volumes by reducedcoordinatesqqqqq.SimulatorBD-TreeThin strandsFast Sphere Update• E.g., surgical sutures (“threads”), hair, etc.• Cosserat model may be more efficient[Pai, Eurographics 02]+ reduces to a 1D problem+ handles geometric non-linearities+ fast (similar to rigid body chain)- useful for thin objects only• Storage cost: 4 floats / mode• Amortized cost: 8 flops / mode• Spheres can be updated in any orderand without access to geometry Output-sensitive collision detectionB725

Integrating Audio Haptic Displays[DiFilippo and Pai UIST 00]User Interaction• Contacts are renderedatomically, with haptics andsound• Same contact forces drive bothhaptics and sound• Haptic force and soundsynchronized to < 1ms• Using– Sound synthesis algorithm of [van denDoel & Pai]– Pantograph haptic device of [Hayward &Ramstein]– Custom DSP controllerThe AHI Audio-HapticInterfaceReality-based Modeling forHaptics and Multimodal DisplaysDinesh K. PaiRutgers UniversityDiFilippo and Pai UIST 00Multisensory integration canimprove haptic perceptionResultsForcee.g., [Doel, Kry and Pai, SIGGRAPH 01]B736

Our Focus:Scanning Contact Interaction BehaviorHow to get models forcontact simulation?• Haptic interaction and, more generally,contact requires multisensory models• Need parametrized models that support lowlatencymultisensory interaction• Need to measure geometry, elasticdeformation, sound response, friction,roughness, etc.Overview• ACME, a robotic facility forscanning contact behavior• Scanningcontact deformation modelscontact texture modelscontact sound modelsACMEThe UBC Active MeasurementFacilityReality-based modelingof Contact[Pai, van den Doel, James,Lang, Lloyd, Richmond, YauSIGGRAPH 2001][Pai,Lang,Lloyd,Woodham ‘99]B747

Preview: Scanning Contact FrictionModel Structure: Green’s FunctionsVideoLinear Elastostatic Model+Reference boundary conditionsGreen’s functions U relateu, vertex displacements top, applied vertex tractionsCan be computed analytically ifmaterial distribution is knowne.g., [JamesPai99, Cotin et al 96,...]• u = Up • =Green’s FunctionsA Framework for ScanningReality-based ModelsModel Structure: Green’s FunctionsModel StructureRenderingMeasurementEstimationLinear Elastostatic Model+Reference boundary conditionsGreen’s functions U relateu, vertex displacements top, applied vertex tractionsCan be computed analytically ifmaterial distribution is knowne.g., [JamesPai99, Cotin et al 96,...]j• u = Ujp • =p jGreen’s FunctionsjMeasurementScanning Deformation BehaviorRobot arm measurescontact force and localdisplacementGlobal displacementmeasured with stereovision and range flowDetails in [LangPai01]B758

Parameter Estimation•Excite vertex j with several p jkestimate vertex displacement u ikfrom range flowParameter EstimationResults•Estimate U ijrobustlyusing TSVD, TLS•Interpolate missingobservationsInteractive Rendering DemoRendered using capacitance matrix algorithmwith haptic force computation at 1KHz[JamesPai99, JamesPai01]B769

RoughnessScanning Contact Texture• Traditionally ≈ small scale variation in surfacegeometry• Our model: small scale variation in friction– Equivalent to traditional model, for frictional contact– Unifies friction and roughness haptic renderingWhat is Contact Texture?Example: Clay Pot• Physical parameters relevant to humanperception of contact [Lederman Klatzky ... ]• Texture = friction + roughness + …• Model should support fast simulation forhaptics (1 KHz) and audio (44.1 KHz)F nF tµFriction• Model: Coulomb FrictionMeasurement:•Easy for small samplef t = -µ | f n | u•Hard for general object: uncertaintyin surface normal, adaptation•We use differential measurementtechnique for robust estimation• Estimate: normal and friction togetherRoughness• Model structure: AR(p) autoregressive modelµ ~e( x)= µ + µ ( x)pµk= ∑ a ~iµk − i+ σεk~i = 1– Captures randomness plus periodicities– Small p sufficient for many surfaces [Thomas 82, Perlin]• Estimate parameters: using covariance method• Rendering: Discrete convolution• Extremely fast and simpleB7710

Real vs. Simulated Clay PotReal µSim µContact Sounds• Provide cues of contact shape, location,force, and object material• “Foley sounds” in radio and cinema• Can be integrated with– Simulation and Interaction[O’BrienCookEssl, DoelKryPai, Friday]– Room acoustics [e.g., Tsingos et al., Friday]Scanning the Clay PotModel Structure• Modal Synthesis [e.g., Doel&Pai 96-01, Cook 96]• Models impulseMresponse at surface point−ditp( x, t) = ∑ ai( x)e sin(2π fit)+i=1+•f i is frequency of a vibration moded i is frequency-dependent damping• a i (x) location dependent amplitude “texture”• Continuous contact sounds generated byconvolution with contact force+Measure Impulse ResponseScanning Contact SoundsB7811

Estimate ParametersScanning the Clay PotFirst estimate modal frequencies f iVideoRefine frequencies + estimate a ik , d i[SteiglitzMcBride65, BrownPuckette93]Conclusionsoriginal90 modes10 modes30 modes• It is now possible to scan multi-modalmodels of contact interaction behavior• Scannable behavior includes– deformation (visual and haptic)– friction and roughness texture– sound response• Can be automated with ACME, a roboticmeasurement facilityRendering• Generate contact force at audio rates– depends on contact texture,nominal force, and velocity[see DoelKryPai paper on Friday]• Convolve with audio impulse response– efficient using modal resonator filter bank(4 flops/mode/sample)– smoothly interpolate audio parameters a i(x) frommesh verticesB7912

Course: Recent Advances in Haptic Rendering and ApplicationsSession V: Novel ApplicationsOutline of Taylor talkApplications inScientific VisualizationRussell TaylorUNC-Chapel Hillhttp://www.cs.unc.edu/~taylorrtaylorr@cs.unc.edu• Eight scientific haptic applications• Three haptic applications in more depth– Listing benefits found from haptics in each– Showing mechanism for coupling haptics thread• (Brief) Haptics for multivariate displayMolecular DynamicsApplication• VMD: Visual Molecular DynamicsHumphrey, 1996Haptic Vector FieldApplication• Lawrence, Lee, Pau, Roman, Novoselov– University of Colorado at Boulder• 5 D.O.F. in• 5 D.O.F. outLawrence, 2000Volume Flow VisualizationUser Study• Iwata and Noma combined HMD and hapticsto display volume data in 1993– Force based on density gradient– or Torque based on density– Either method improved positional accuracy• Describe presenting flow field– Force for flow velocity– Torque for vorticityIwata, 1993B80

Medical Volume ExplorationApplication• Avila and Sobierajski 1996– Feeling neuron dendrites• Repulsion was too hard to followfor twisting dendrites– Gentle attraction used• Opposite of gradientVector and Tensor FieldApplications• The Visual Haptic WorkbenchConstrain to streamlineAnisotropic dragAvila, 1996Ikits, 2005When is Force Useful?UNC Experiences• Teaching physics potential fields• Drug/protein docking• nanoManipulatorForce-Field Simulation forTraining• 2D sliding carriage device• Presented magnetic, electric, andgravitational fields• Motivated students learned more– Less motivated students didn’t• Dispelled misconceptions– Electric field in diode is not greaternear the plate than near the cathode– Gravity field in 3-body system does notalways point towards one of the bodiesDrug/Protein Docking• Ming Ouh-Young– 6-DOF positioning task– “Lock and Key” problem– Hard surface + electrostatic– Got a better “feel” for docking– User study: NTE factor-of-2 speedup with hapticsBrooks, 1990Brooks, 1990B81

nanoManipulatornanoManipulator: HapticsA virtual environmentinterface to SPMThe Goal:• Remove boundariesbetween user and sample• Can we make experimentson the molecular scale aseasy as rolling a pencil orpushing a golf ball?“It was really a remarkable feeling for a chemist tobe running his hand over atoms on a surface”– R. Stanley Williams, UCLA Chemistry• Exciting and engaging, but what is it useful for?Finding the right spotFinding the right pathPositionwhenscanningPositionafter being heldstill for several secondsAlso, finding thetop of anadenovirusLight touch (haptic imaging)Observation modifies the systemB82

Case Study ofMulti-ThreadingnManipulator: The Old Way• Problem: Haptics must update at >500 Hz– Required for stable hard surfaces– Must be uninterrupted to prevent force discontinuities• Solution: As Ken & Ming described– Separate force-feedback server– Pass Intermediate representation between threadsForce-FeedbackDevicePositionMove Microscope Tip, Read HeightSend Force (Depends on Height)Draw ImageForceVideo11 Server measures end effector 22 User interface moveslocationmicroscope tip to track endeffector motionSPMUser InterfacePhantomcontrols force andmicroscope44Decoupling surface updateusing intermediate rep.Plane is transformed intodevice coordinates andpresented to user33Two samples + “up” yield atangent plane to the surface atcontact pointLocal Plane EquationProbeSurfaceLocal PlaneApproximation(Used in Force Loop)Complete Surface(known toApplication)Preventing Discontinuity• Discontinuity when plane updatedPreventing Discontinuity• Recovery time over several stepsProbet = 0 t > 0Probe is beneaththe updated planeProbet = 0 t > 0Probe is broughtgradually toupdated planeResult: Large force10ms update enabled10X stable stiffnessB83

SEM+AFM: See while touch• SEM view of AFM tip• Hand-controlled AFM• Zooms in on nanotube• “Twangs” nanotube3D Magnetic ForceMicroscopeSandpaper: Haptic TexturesAugmenting “Basic Haptics”• Margaret Minsky dissertation (MIT, UNC)• Displayed 3D surfaces on 2D haptic device bymapping slope to lateral force– People perceived 3D shape• Displayed several properties– Viscosity– Spatial Frequency• Vary properties based on data• Jason Fritz, University of Delaware– Adding haptic “grid planes” provide scale– Friction and texture• Produce more realistic-feeling surfaces• Can distinguish features in data sets– Produced stochastic texture model• Creates information-rich haptic texturesMinsky, 2000Fritz, 1996B84

Haptics for Multi-Dimensional Display• Modulate surface properties– friction, stiffness, bumpiness, vibration, adhesion• User studies done by Mark Hollins at UNC todetermine the perceptually-linear mapping for each– Map directly for the relevant physical parameter– Linearize for presentation of non-physical quantities• Ongoing exploration of cross-talk between channelsSeeger, 2000Summary• Useful for– Direct perception of force fields– Sensing while manipulating simulation or tele-operation– Tracing without occlusion in 3D• Match application requirements– 2DOF in/out for force fields– 6DOF in/out for Docker– 6DOF in, 3DOF out for nM• Separate force, graphics and control loopsnanoManipulator Co-builders(78)WPAFBBelgiumTorontoASUNISTPsychologyCS Graphics3rdTechEducationCS ImageInformation & Library ScienceCS Dist. Sys.PhysicsBiologyGene TherapyChemistryNIEHSRTPSupport provided byReferences– 3DMFM• NIH National Institute for Biomedical Imaging andBioengineering• NIH National Center for Research Resources• National Science Foundation• Army Research Office• Office of Naval Research• Avila, R. S. and L. M. Sobierajski (1996). “A Haptic Interaction Method forVolume Visualization,” Proceedings of IEEE Visualization '96, San Francisco.• Brooks, F. P., Jr., M. Ouh-Young, et al., “Project GROPE - Haptic displays forscientific visualization,” Computer Graphics: Proceedings of SIGGRAPH '90,Dallas, Texas. 177-185. 1990.• Fritz, J. and K. Barner (1996). Stochastic models for haptic texture.Proceedings of the SPIE International Symposium on Intelligent Systems andAdvanced Manufacturing - Telemanipulator and Telepresence TechnologiesIII, Boston, MA.• William Humphrey, Andrew Dalke, and Klaus Schulten, VMD - VisualMolecular Dynamics,” Journal of Molecular Graphics, 14:33-38, 1996.• Milan Ikits and J. Dean Brederson, “The Visual Haptic Workbench,”Visualization Handbook, eds. Charles Hansen and Christopher Johnson,Elsevier, 2005B85

References• Iwata, H.; Noma, "Volume haptization", 1993. Proceedings., IEEE 1993Symposium on Research Frontiers in Virtual Reality, pp. 16 -23, 25-26 Oct.1993.• D. A. Lawrence, C. D. Lee, L. Y. Pao, and R. Y. Novoselov. "Shock andVortex Visualization Using a Combined Visual/Haptic Interface," Proc. IEEEConference on Visualization and Computer Graphics, Salt Lake City, UT, Oct.2000.• Minsky, M., M. Ouh-young, et al., “Feeling and Seeing: Issues in ForceDisplay,” Proceedings ACM Siggraph Symposium on Interactive 3D Graphics.2000.• Seeger, A., A. Henderson, et al., “Haptic Display of Multiple Scalar Fields on aSurface,” Workshop on New Paradigms in Information Visualization andManipulation, ACM Press. 2000.B86

Course: Recent Advances in Haptic Rendering and ApplicationsSession V: Novel ApplicationsGoalHaptic Interactionwith Fluid Media• Enable users to FEEL simulated fluidWilliam Baxter and Ming C. LinDepartment of Computer ScienceUniversity of North Carolina at Chapel Hillhttp://www.cs.unc.edu/~lin{baxter,lin}@cs.unc.eduSensAble’s PhantomGoalGoalHaptics OverviewMotivation• Programmatic force feedback I/O• Input– 6DOF – Position & Rotation• Output– 3DOF – Forces– 6DOF – Forces & Torques• Need kHz updatesSensAble’s Phantom– For smooth output• Allow users to feel and touchVirtual RealitySimulation / TrainingScientific VisualizationInteractive ApplicationsB87

Motivation: Virtual RealityMotivation: ScientificVisualization• Immersive visualization• Computational steeringHaptics add to the illusionof presence in VR[Insko 2001]Motivation: Simulation/TrainingInteractive flight simulationHaptic cues for reducingSpatial DisorientationMotivationEntertainmentEducationFluid-based paintingMotivation: Interactive Painting• dAb [SIGGRAPH ’01]• IMPaSTo [NPAR ’04]• Stokes Paint [CASA ’04]• Sim Brush [PG ’04]Main Results• Physically based force computation modelfor haptic interaction with fluid media– Based on stress tensor formulation– Low computational cost• Haptic filtering technique– Based on FIR filter designB88

Organization• Previous work• Overview• Haptic Force Computation• Haptic Filtering• ResultsOrganization• Previous work• Overview• Haptic Force Computation• Haptic Filtering• ResultsPrevious Work• Fluid-Structure Interaction OfflineSimulation– Foster & Metaxas ’96– Ristow ’99,’00– Carlson et al. ’04Previous Work• Haptic Rendering– Point interaction with rigid bodiesHollerbach et al. ’97, Nahvi et al. ’98– Point interaction with deformable bodiesAyache & Delingette ’03– 6DOF interaction with rigid bodiesGregory ’00– Haptic texture renderingSiira & Pai ’96, Otaduy ’04Previous Work• Haptics of Vector Fields– Helman & Hesselink ’90,’91– Durbeck et al. ’98– Lawrence et al. ’00– Mascarenhas et al. ’01Organization• Previous work• Overview• Haptic Force Computation• Haptic Filtering• ResultsB89

Navier-Stokes EquationsOverview• Update virtualprobe positionMomentum equationConservation of mass• Output:u velocity andp pressure• Obtain u and p• Compute force• Output forceOverview – Online Simulation• Update virtualprobe position• Apply boundaryconditions• Simulate fluid• Compute force• Output forceTwo-way couplingOverview – OfflineSimulation• Update virtualprobe position• Retrieve offlinesimulation data• Compute force• Output forceOne-way couplingOrganization• Previous work• Overview• Haptic Force Computation• Haptic Filtering• ResultsCommon ForceApproximations• Aerodynamic drag• Viscous DragvAB90

Fluid Force ComputationFluid Force Computation• Fluid stress tensorpressure viscosity• Force/AreapPn µP µnp• Integrate over surface– Force– TorquePnrrPnBuoyancy?Discretization• Incorporated automatically∑P i∆Agppp pppppFPnrrPnGrid-based SolutionGrid-based Solution• Boundary approximationAll normals, n, axis-aligned• One-sided finite difference derivativesB91

Organization• Previous work• Overview• Haptic Force Computation• Haptic Filtering• ResultsHaptic Display• Problem:- Sim Generates30-100Hz data- Want 1000Hz output• One Solution:-Filtering• Goal: smooth out abrupt changesSensAble’s PhantomFiltering• Remove discontinuities• Finite Impulse Response• Bartlett-Hanning window filterFiltering - Reconstruction• Analogous to image reconstructionFILTERFILTERFilteringOrganization• Previous work• Overview• Haptic Force Computation• Haptic Filtering• ResultsB92

Results• 2D Navier-Stokes solverResults– 3D Stokes fluid painting simulation(Baxter et al CASA 2004)Summary• First fluid-haptic interaction• Effective, inexpensive filtering• Integration with interactive paintingConclusion• Can enhance VR/Sim Apps– Users feel enriched experience• General approach– Applies to both online & offline simulation• Force computation is affordable– Bottleneck is fluid solver, not forceLimitations• Sampling of fluid solution– Many numerical methods have poor accuracyaround boundary• Filtering Forces– Can’t create new informationFuture Work• Higher order boundary treatment– e.g. GENSMAC• Other interactive fluid methods (non-gridmethods)– Smoothed Particle Hydrodynamics (SPH)– Moving Particle Semi-Implicit (MPS)• Analytical requirements for smooth force output• Criteria for force filtersB93

AcknowledgementsThe End• Link Fellowship• NVIDIA Fellowship• Army Research Office• Intel Corporation• National Science Foundation• Office of Naval ResearchFor more information, seehttp://gamma.cs.unc.edu/interactivehttp://gamma.cs.unc.edu/DABhttp://gamma.cs.unc.edu/HAPTIC_FLUID/B94