download LR pdf - Kabk

Underlying TechnologyScreens and GlassesUnlike screen-based AR, HMDs provide depthperception as both eyes receive an image.When objects are projected on a 2D screen,one can convey an experience of depth byletting the objects move. Recent 3D screensallow you to view stationary objects in depth.3D televisions that work with glasses quicklyalternate the right and left image - in sync withthis, the glasses use active shutters which letthe image in turn reach the left or the righteye. This happens so fast that it looks like youview both, the left and right image simultaneously.3D television displays that work withoutglasses make use of little lenses which areplaced directly on the screen. Those refractthe left and right image, so that each eye canonly see the corresponding image. See forexample www.dimenco.eu/display-technology.This is essentially the same method as usedon the well known 3D postcards on which abeautiful lady winks when the card is slightlyturned. 3D film makes use of two projectorsthat show the left and right images simultaneously,however, each of them is polarized in adifferent way. The left and right lenses of theglasses have matching polarizations and onlylet through the light of to the correspondingprojector. The important point with screensis that you are always bound to the physicallocation of the display while headset basedtechniques allow you to roam freely. This iscalled immersive visualization — you are immersedin a virtual world. You can walk aroundin the 3D world and move around and entervirtual 3D objects.Video-See-Through AR will become popularwithin a very short time and ultimately becomean extension of the smartphone. This isbecause both display technology and cameratechnology have made great strides with theadvent of smartphones. What currently stillmight stand in the way of smartphone modelsis computing power and energy consumption.Companies such as Microsoft, Google, Sony,Zeiss,... will enter the consumer market soonwith AR technology.Tracking TechnologyA current obstacle for major applications whichsoon will be resolved is the tracking technology.The problem with AR is embedding the virtualobjects in the real world. You can comparethis with color printing: the colors, e.g., cyan,magenta, yellow and black have to be printedproperly aligned to each other. What youoften see in prints which are not cut yet, areso called fiducial markers on the edge of theprinting plates that serve as a reference forthe alignment of the colors. These are alsonecessary in AR. Often, you see that markersare used onto which a 3D virtual object isprojected. Moving and rotating the marker, letsyou move and rotate the virtual object. Sucha marker is comparable to the fiducial markerin color printing. With the help of computervision technology, the camera of the headsetcan identify the marker and based on it’s size,shape and position, conclude the relative positionof the camera. If you move your head relativeto the marker (with the virtual object), thecomputer knows how the image on the displaymust be transformed so that the virtual objectremains stationary. And conversely, if yourhead is stationary and you rotate the marker,it knows how the virtual object should rotateso that it remains on top of the marker.AR smartphone applications such as Layar usethe build in GPS and compass for the tracking.This has an accuracy of meters and measuresangles of 5-10 degrees. Camera-based tracking,however, is accurate to the centimetre and canmeasure angles of several degrees. Nowadays,using markers for the tracking is already outof date and we use so called “natural featuretracking” also called “keypoint tracking”.Here, the computer searches for conspicuous(salient) key points in the left and right cameraimage. If, for example, you twist your head, thisshift is determined on the basis of those key pointswith more than 30 frames per second. This way, a3D map of these keypoints can be built and the computerknows the relationship (distance and angle)between the keypoints and the stereo camera. Thismethod is more robust than marker based trackingbecause you have many keypoints — widely spreadin the scene — and not just the four corners of themarker close together in the scene. If someonewalks in front of the camera and blocks some of thekeypoints, there will still be enough keypoints leftand the tracking is not lost. Moreover, you do nothave to stick markers all over the world.Collaboration with the Royal Academy of Arts(KABK) in The Hague in the AR Lab (RoyalAcademy, TU Delft, Leiden University, variousSMEs) in the realization of applications.The TU Delft has done research on AR since 1999.Since 2006, the university works with the artacademy in The Hague. The idea is that AR is a newtechnology with its own merits. Artists are verygood at finding out what is possible with the newtechnology. Here are some pictures of realizedprojects. liseerde projectenFig 1. The current technology that replaces the markerswith natural feature tracking or so called keypointtracking. Instead of the four corners of the marker, thecomputer itself determines which points in the left andright images can be used as anchor points for calculatingthe 3D pose of the camera in 3D space. From top:1: you can use all points in the left and right imagesto slowly build a complete 3D map. Such a map can,for example, be used to relive your past experiencebecause you can again walk in the now virtual space.2: the 3D keypoint space and the trace of thecamera position within it.3: keypoints (the color indicates the suitability)4: you can place virtual objects (eyes) on an existingsurface2223