Augmented Reality: A class of displays on the reality-virtuality ...

3. TWO CATEGORIES OF AUGMENTED REALITY DISPLAYSWithin the context <strong>of</strong> Fig. 1, the above-mentioned broad definition <strong>of</strong> <strong>Augmented</strong> <strong>Reality</strong> – "augmentingnatural feedback to the operator with simulated cues" – is quite clear. Also noteworthy in this figure is thecorresponding concept <strong>of</strong> <strong>Augmented</strong> Virtuality (AV), which results automatically, both conceptually andlexically, from the figure. Some examples <strong>of</strong> AV systems are given below, in Section 4. In the presentsection we first contrast two cases <strong>of</strong> AR, those based on head-mounted see-through <strong>displays</strong> and thosewhich are monitor based, but both <strong>of</strong> which comply with the definition depicted in Fig. 1.3.1 "See-through" AR <strong>displays</strong>This <strong>class</strong> <strong>of</strong> <strong>displays</strong> is characterised by the ability to see through the display medium directly to the worldsurrounding the observer, thereby achieving both the maximal possible extent <strong>of</strong> presence and the ultimatedegree <strong>of</strong> "realspace imaging". 3 Most commonly display augmentation is achieved by using mirrors tosuperimpose computer generated graphics optically onto directly viewed real-world scenes. Such <strong>displays</strong>are already a mature technology in some (mostly military) aviation systems, as either panel-mounted orhead-mounted <strong>displays</strong> (HMD's), but are currently finding new applications as a VR related technology,especially in manufacturing and maintenance environments. 4,5,6 Of special interest are recent efforts toapply such AR <strong>displays</strong> to medical imaging, by superimposing data acquired via imaging techniques suchas ultrasound, CT scanning, etc. conformally onto the actual patient. 7,8Several research and development issues have accompanied the advent <strong>of</strong> optical see-through (ST) <strong>displays</strong>.These include the need for accurate and precise, low latency body and head tracking, accurate and precisecalibration and viewpoint matching, adequate field <strong>of</strong> view, and the requirement for a snug (no-slip) butcomfortable and preferably untethered head-mount. 5,9 Other issues which present themselves are moreperceptual in nature, including the conflicting effects <strong>of</strong> occlusion <strong>of</strong> apparently overlapping objects andother ambiguities introduced by a variety <strong>of</strong> factors which define the interactions between computergenerated images and real object images. 9 Perceptual issues become even more challenging when ST-ARsystems are constructed to permit computer augmentation to be presented stereoscopically. 10Some <strong>of</strong> these technological difficulties can be partially alleviated by replacing the optical ST with aconformal video-based HMD, thereby creating what is known as "video see-through". Such <strong>displays</strong>present certain advantages, both technological and perceptual 9 , even as new issues arise from the need tocreate a camera system whose effective viewpoint is identical to that <strong>of</strong> the observer's own eyes. 113.2 Monitor based AR <strong>displays</strong>We use the term monitor-based (non-immersive), or "window-on-the-world" (WoW), AR to refer todisplay systems where computer generated images are either analogically or digitally overlaid onto live orstored video images. 12,13,14 Although the technology for achieving this has been well-known for sometime, most notably by means <strong>of</strong> chroma-keying, a large number <strong>of</strong> useful applications present themselveswhen this concept is implemented stereoscopically. 15,16,17In our own laboratory this <strong>class</strong> <strong>of</strong> monitor-based AR <strong>displays</strong> has been under development for some years,as part <strong>of</strong> the ARGOS (<strong>Augmented</strong> <strong>Reality</strong> through Graphic Overlays on Stereovideo) project. 18 Severalstudies have been carried out to investigate the practical applicability <strong>of</strong>, among other things, overlaidstereographic virtual pointers and virtual tape measures 19 , virtual landmarks 20 , and virtual tethers 21 fortelerobotic control. Current efforts are focused on applying more advanced stereographic tools for achievingvirtual control 22,23,24 <strong>of</strong> telerobotic systems, through the use <strong>of</strong> overlaid virtual robot simulations, virtualencapsulators, and virtual bounding planes, etc. 25284 / SPIE Vol. 2351, Telemanipulator and Telepresence Technologies (1994)