download LR pdf - Kabk

Figure 2. Head mounted display, modifiedCinemizer OLED (Carl Zeiss) with two MicrosoftHD-5000 webcams.Figure 3. Sparse 3D feature map generated bythe pose estimation module.Figure 4. Graphical user interface options menu.ated reality system supports a lightweight headmounteddisplay (HMD), contactless augmentationalignment, and a remote connection to and collaborationwith expert crime scene investigators.The video see-through of a modified Carl ZeissCinemizer OLED (cf. Figure 2) for displayingcontent fulfills the requirement for a lightweightHMD, as its total weight is ~180 grams. TwoMicrosoft HD-5000 webcams are stripped andmounted in front of the Cinemizer providing afull stereoscopic 720p resolution pipeline. Bothcameras record at ~30hz in 720p, images areprojected in our engine, and render 720p stereoscopicimages to the Cinemizer.As for all mediated reality systems, robust realtimepose estimation is one of the most crucialparts, as the 3D pose of the camera in the physicalworld is needed to render virtual objectscorrectly on required positions. We use a heavilymodified version of PTAM (Parallel Tracking andMapping) (Klein and Murray, 2007), in whicha single camera setup is replaced by a stereocamera setup using 3D natural feature matchingand estimation based on natural features. Usingthis algorithm, a sparse metric map (cf. Figure3) of the environment is created. This sparsemetric map can be used for pose estimation inour Augmented Reality system.In addition to the sparse metric map, a dense3D map of the crime scene is created. Thedense metric map provides a detailed copy ofthe crime scene enabling detailed analysis andis created from a continuous stream of disparitymaps that are generated while the user movesaround the scene. Each new disparity map isregistered (combined) using the pose informationfrom the PE module to construct or extendthe 3D map of the scene. The point clouds areused for occlusion and collision checks, and forsnapping digital objects to physical locations.By using an innovative hand tracking system,the mediated reality system can recognize barehands gestures for user interface operation.This hand tracking system utilizes the stereocamera rig to detect the hand movements in 3D.The cameras are part of the HMD and an adaptivealgorithm has been designed to determinewhether to rely on the color, disparity or on bothdepending on the lighting conditions. This is thecore technology to fulfill the requirement ofbare hand interfacing. The user interface andthe virtual scene are general-purpose parts ofthe mediated reality system. They can be usedfor CSI, but also for any other mediated realityapplication. The tool set, however, needs to betailored for the application domain. The currentmediated reality system supports the followingtasks for CSIs: recording the scene, placing tags,loading 3D models, bullet trajectories and placingrestricted area ribbons. Figure 4 shows thecorresponding menu attached to a user’s hand.The mediated reality system has been evaluatedon a staged crime scene at the NFI’s Labwith three observers, one expert and onelayman with only limited background in CSI.Within the experiment the layman, facilitatedby the expert, conducted three spatial tasks,i.e. tagging a specific part of the scene withinformation tags, using barrier tape and polesto spatially secure the body in the crime sceneand analyzing a bullet trajectory analysis withricochet. The experiment was analyzed alongseven dimensions (Burkhardt et al., 2007): fluidityof collaboration, sustaining mutual understanding,information exchanges for problemsolving, argumentation and reaching consensus,task and time management, cooperative orientation,and individual task orientation. Theresults show that the mediated reality systemsupports remote spatial interaction with thephysical scene as well as collaboration in sharedaugmented space while tackling current issues incrime scene investigation. The results also showthat there is a need for more support to identifywhose turn it is and who wants the next turn,etc. Additionally, the results show the need torepresent the expert in the scene to increasethe awareness and trust of working in a teamand to counterbalance the feeling of being observed.Knowing the expert’s focus and currentactivity could possibly help to overcome this issue.Whether traditional patterns for computermediatedinteraction (Schümmer and Lukosch,2007) support awareness in mediated realityor rather new forms of awareness need to bedesigned, will be the subject of future research.Further tasks for future research include thedesign and evaluation of alternative interactionpossibilities, e.g. by using physical objects thatare readily available in the environment, sensorfusion, image feeds from spectral cameras orpreviously recorded laser scans, to provide moresituational awareness and the privacy, securityand validity of captured data. Finally, thoughIT is being tested and used for educationalpurposes within the CSI Lab of the NetherlandsForensic Institute (NFI), only the application andtest of the mediated reality system in real settingscan show the added value for crime sceneinvestigation.REFERENCES■ R. Azuma, A Survey of Augmented Reality,Presence 6, Vol 4, 1997, 355-385■ J. Burkhardt, F. Détienne, A. Hébert, L. Perron,S. Safin, P. Leclercq, An approach to assess thequality of collaboration in technology-mediateddesign situation, European Conference on CognitiveErgonomics: Designing beyond the Product - UnderstandingActivity and User Experience in UbiquitousEnvironments, 2009, 1-9■ G. Klein, D. Murray, Parallel Tracking and Mappingfor Small AR Workspaces, Proc. InternationalSymposium on Mixed and Augmented Reality, 2007,225-234■ S. Mann, W. Barfield, Introduction to MediatedReality, International Journal of Human-ComputerInteraction, 2003, 205-208■ R. Poelman, O. Akman, S. Lukosch, P. Jonker, Asif Being There: Mediated Reality for Crime SceneInvestigation, CSCW ‘12: Proceedings of the 2012ACM conference on Computer Supported CooperativeWork, ACM New York, NY, USA, 2012, 1267-1276,http://dx.doi.org/10.1145/2145204.2145394■ T. Schümmer, S. Lukosch, Patterns for Computer-Mediated Interaction, John Wiley & Sons, Ltd. 20077475