Human-Computer Interaction and Presence in Virtual Reality

Human-Computer Interaction and Presence in
Virtual Reality Exposure Therapy

Human-Computer Interaction and Presence in
Virtual Reality Exposure Therapy
Proefschrift
ter verkrijging van de graad van doctor
aan de Technische Universiteit Delft,
op gezag van de Rector Magnificus prof.dr.ir. J.T. Fokkema,
voorzitter van het College voor Promoties,
in het openbaar te verdedigen op maandag 3 maart 2003, om 13:30 uur
door Martijn Jeroen SCHUEMIE
docterandus in de economie
geboren te Vught

Dit proefschrift is goedgekeurd door de promotor:
Prof.dr.ir. F.W. Jansen
Toegevoegd promotor:
Dr.ir. C.A.P.G. van der Mast
Samenstelling promotiecommissie:
Rector Magnificus, voorzitter
Prof.dr.ir. F.W. Jansen, Technische Universiteit Delft, promotor
Dr.ir. C.A.P.G. van der Mast, Technische Universiteit Delft, toegevoegd promotor
Prof.dr. M.A. Neerincx, Technische Universiteit Delft
Prof.dr.ir. J.L.G. Dietz, Technische Universiteit Delft
Prof.dr. H. de Ridder, Technische Universiteit Delft
Prof.dr. P.M.G. Emmelkamp, Universiteit van Amsterdam
Prof.dr. J.D. Foley, Georgia Institute of Technology
SIKS Dissertation series No. 2003-03
The research reported in this thesis has been carried out under the auspices of SIKS, the
Dutch Research School for Information and Knowledge Systems.
ISBN 90-6734-166-5

Contents
CONTENTS .......................................................................................................................... V
ACKNOWLEDGEMENTS ................................................................................................IX
1 INTRODUCTION .............................................................................................................. 1
1.1 Virtual Reality.................................................................................................................... 1
1.2 Presence............................................................................................................................. 2
1.3 A new class of applications................................................................................................ 3
1.4 Virtual Reality Exposure Therapy...................................................................................... 4
1.5 Research goal .................................................................................................................... 5
1.6 Dissertation outline............................................................................................................ 6
2 RESEARCH APPROACH................................................................................................. 7
2.1 Design method ................................................................................................................... 7
2.1.1 Existing HCI design methods.................................................................................... 8
2.1.2 VRET Design method............................................................................................... 9
2.1.3 Evaluation ................................................................................................................. 9
2.2 Presence model................................................................................................................ 10
2.2.1 Evaluation ............................................................................................................... 10
2.3 Patient and therapist UI .................................................................................................. 11
2.3.1 Evaluation ............................................................................................................... 11
2.3.2 Determining usability.............................................................................................. 12
2.3.3 Determining usability in VRET .............................................................................. 13
2.3.4 Evaluation of the patient UI .................................................................................... 14
2.3.5 Evaluation of the therapist UI ................................................................................. 14
2.3.6 Further evaluation ................................................................................................... 15
2.4 Conclusions...................................................................................................................... 16
3 ANALYSIS ........................................................................................................................ 17
3.1 A Brief History of VR Exposure Therapy......................................................................... 17
3.1.1 Fear of Heights........................................................................................................ 17
3.1.2 Fear of Flying.......................................................................................................... 18
3.1.3 Agoraphobia............................................................................................................ 19
3.1.4 Arachnophobia........................................................................................................ 19
3.1.5 Fear of public speaking ........................................................................................... 19
3.1.6 Claustrophobia ........................................................................................................ 19
3.1.7 Conclusions............................................................................................................. 20
3.2 Current Task Analysis methods ....................................................................................... 20
3.2.1 Modeling tasks ........................................................................................................ 21
3.2.2 Modeling user cognition ......................................................................................... 23
3.2.3 Modeling the user interface..................................................................................... 24
3.2.4 Applying current Task Analysis methods to VRET................................................ 25
3.3 Adapting existing TA methods ......................................................................................... 27
3.3.1 Interaction cycle...................................................................................................... 28
3.3.2 Procedural information............................................................................................ 28
v

3.3.3 State information..................................................................................................... 29
3.3.4 Overview................................................................................................................. 29
3.4 Gathering data................................................................................................................. 30
3.4.1 HCI Data gathering techniques ............................................................................... 30
3.4.2 Data gathering in VRET.......................................................................................... 31
3.5 Current VRET practice .................................................................................................... 32
3.5.1 Current state-of-the-art............................................................................................ 32
3.5.2 Virtual Reality testbed............................................................................................. 34
3.5.3 Case: acrophobia treatment ..................................................................................... 35
3.5.4 Interpreting observations......................................................................................... 36
3.5.5 Identifying sequence ............................................................................................... 38
3.5.6 Conclusions............................................................................................................. 40
3.6 Current In Vivo therapy................................................................................................... 41
3.6.1 Current In Vivo practice.......................................................................................... 41
3.7 Task Model....................................................................................................................... 42
3.7.1 High-level goals ...................................................................................................... 43
3.7.2 Media ...................................................................................................................... 43
3.7.3 Task decomposition ................................................................................................ 44
3.7.4 Functional requirements.......................................................................................... 49
3.7.5 Informational requirements..................................................................................... 50
3.7.6 Usability problems in VRET................................................................................... 52
3.8 Conclusions...................................................................................................................... 53
4 PRESENCE....................................................................................................................... 55
4.1 On the nature of presence................................................................................................ 55
4.1.1 Definitions...............................................................................................................55
4.1.2 Theories................................................................................................................... 57
4.1.3 Conclusions on the nature of presence.................................................................... 59
4.2 Impact of presence........................................................................................................... 59
4.2.1 Subjective sensation................................................................................................ 59
4.2.2 Task performance.................................................................................................... 60
4.2.3 Responses and emotions ......................................................................................... 60
4.2.4 Simulator sickness................................................................................................... 61
4.2.5 Conclusions on results of presence ......................................................................... 62
4.3 Measuring presence......................................................................................................... 62
4.3.1 Subjective measures: Questionnaires ...................................................................... 62
4.3.2 Other subjective measures....................................................................................... 67
4.3.3 Objective measures: Behavioral.............................................................................. 67
4.3.4 Objective measures: Physiological.......................................................................... 68
4.3.5 Conclusions on measures ........................................................................................ 68
4.4 Causes of presence........................................................................................................... 69
4.4.1 VR characteristics ................................................................................................... 70
4.4.2 User characteristics ................................................................................................. 74
4.4.3 Conclusions on causes of presence ......................................................................... 75
4.5 The role of presence in VRET .......................................................................................... 76
4.6 HCI and the effectiveness of VRET.................................................................................. 77
4.6.1 The PGMC model ................................................................................................... 78
4.7 Conclusions...................................................................................................................... 79
vi

5 PATIENT'S USER INTERFACE ................................................................................... 81
5.1 Requirements ................................................................................................................... 81
5.2 Design options ................................................................................................................. 82
5.2.1 Input devices ........................................................................................................... 83
5.2.2 Output devices......................................................................................................... 84
5.2.3 Locomotion techniques ........................................................................................... 84
5.2.4 Locomotion in immersive VR................................................................................. 85
5.2.5 Evaluations.............................................................................................................. 86
5.2.6 Conclusions............................................................................................................. 88
5.3 Experiment 1: Patient vs. therapist control ..................................................................... 88
5.3.1 Method ....................................................................................................................89
5.3.2 Results..................................................................................................................... 91
5.3.3 Discussion ............................................................................................................... 92
5.4 Experiment 2: Patient vs. therapist control ..................................................................... 93
5.4.1 Method ....................................................................................................................94
5.4.2 Results..................................................................................................................... 96
5.4.3 Discussion ............................................................................................................... 99
5.5 Experiment 3: Patient Control....................................................................................... 100
5.5.1 Method .................................................................................................................. 100
5.5.2 Results................................................................................................................... 104
5.5.3 Discussion ............................................................................................................. 107
5.6 Discussion...................................................................................................................... 107
5.7 Conclusions.................................................................................................................... 108
6 THERAPIST'S USER INTERFACE............................................................................ 111
6.1 Requirements ................................................................................................................. 111
6.2 Design options ............................................................................................................... 111
6.2.1 SUDs recording..................................................................................................... 113
6.2.2 Free viewpoint....................................................................................................... 113
6.2.3 Automatic pilot ..................................................................................................... 114
6.2.4 Graphical controls................................................................................................. 115
6.3 Experiment 4: Exploratory evaluation........................................................................... 115
6.3.1 Method .................................................................................................................. 115
6.3.2 Results................................................................................................................... 116
6.3.3 Discussion ............................................................................................................. 116
6.4 Experiment 5: Patient vs. Therapist control .................................................................. 117
6.4.1 Method .................................................................................................................. 117
6.4.2 Results................................................................................................................... 118
6.4.3 Discussion ............................................................................................................. 121
6.5 Experiment 6: Free viewpoint........................................................................................ 122
6.5.1 Method .................................................................................................................. 122
6.5.2 Results................................................................................................................... 123
6.5.3 Discussion ............................................................................................................. 124
6.6 Discussion...................................................................................................................... 124
6.7 Conclusions.................................................................................................................... 125
7 CONCLUSIONS ............................................................................................................. 127
7.1 Task Analysis method..................................................................................................... 127
7.1.1 Instructing the computer ....................................................................................... 127
vii

7.1.2 Determining the patient's fear ............................................................................... 129
7.1.3 Usefulness of the TA method................................................................................ 129
7.2 Presence model.............................................................................................................. 130
7.2.1 Therapist versus patient control ............................................................................ 130
7.2.2 Patient locomotion technique................................................................................ 131
7.2.3 Usefulness of the presence model ......................................................................... 131
7.3 User Interface design for VRET..................................................................................... 132
7.4 Limitations ..................................................................................................................... 133
7.5 Future research.............................................................................................................. 133
7.6 General conclusions ...................................................................................................... 134
REFERENCES .................................................................................................................. 135
ABBREVIATIONS............................................................................................................ 149
APPENDIX A: QUESTIONNAIRES............................................................................... 151
A.1 Task Analysis................................................................................................................. 151
A.2 Experiment 1 ................................................................................................................. 152
A.3 Experiment 2 ................................................................................................................. 153
A.4 Experiment 3 ................................................................................................................. 154
A.5 Experiment 4 ................................................................................................................. 155
A.6 Experiment 5 ................................................................................................................. 156
A.7 Experiment 6 ................................................................................................................. 158
APPENDIX B: INFORMATION NEEDS....................................................................... 159
SAMENVATTING ............................................................................................................ 165
CURRICULUM VITAE.................................................................................................... 169
SIKS DISSERTATIONS................................................................................................... 171
INDEX ................................................................................................................................ 175
viii

Acknowledgements
The research as described in this thesis was part of a truly multi-disciplinary project,
involving both human-computer interaction and clinical psychology. I think both disciplines
gained from this cooperation, and I hope it will continue into the distant future.
It has given me much pleasure working together with a wide variety of people, and I would
like to acknowledge them here, although I am bound to forget one or two.
First and foremost, I would like to thank my supervisor and co-promotor Charles van der
Mast for his excellent guidance during this research and also for making the research
possible in the first place. If it wasn’t for Charles, the Virtual Reality and Phobias project
would not have existed.
Furthermore I would like to thank Paul Emmelkamp and Merel Krijn of the University of
Amsterdam for their excellent cooperation.
Many thanks to Peter van der Straaten and Bas van Abel, who assisted me in my research.
Also thanks to Lucas van Gerwen, Johan van Luik, Josine Arondeus and Imke Grootenhuis
of the VALK foundation. Thanks to the technical support staff of both departments two and
five of the ITS faculty, to Jan Dietz, Philip Spinhoven, Sietske de Vries, Lex Hulsbosch,
Roeline Biemong, Claudius de Wilde de Ligny, Hanneke Verheul, Marthe van der Pol and
Almuth Wilmsmeyer. To my ‘roommates’ Abraham Guyt and Laura Zondervan for
providing the occasional distraction. To Jan van der Berg for starting my interest in a career
in science, and to my promotor Erik Jansen for his suggestions and comments which have
greatly improved this thesis.
ix

1 Introduction
Human Computer Interaction (HCI) is concerned with the way humans interact with
technology. It deals with how humans work with computers and how computer systems can
be designed to best facilitate the users in achieving their goals. There is a gap to be bridged,
with computer technology on the one side, and the human operator on the other side. In these
times we see rapid technological advancements, in terms of computer performance, everincreasing
telecommunication possibilities and new and improved interface devices such as
lightweight LCD displays and magnetic and optical tracking devices.
With the progression of technology over the years we have also seen improvements in the
interface through which the user interacts with a computer system: the User Interface (UI).
The first of what can be called the modern computers, such as the ENIAC (McCartney,
2001) required users to program the computer by connecting cables on a patchboard. Later, a
less cumbersome UI was possible in the shape of punch tapes or cards that were used as in-
and output. The development of computer screens with the possibility of displaying text
opened the way for more diversity in the UI, allowing the user to interact with the computer
using a command driven or menu driven interface. Advances in display technology to the
point where graphics could be displayed on the screen resulted in different types of user
interfaces. One of these was the Sketchpad developed by Ivan Sutherland (Sutherland, 1963),
where a user could directly manipulate objects on the screen using a light pen.
Simultaneously, other input devices such as the mouse were developed. Eventually the UI
paradigm most used today was born: the WIMP interface (Window, Icon, Menus and
Pointer), developed not more than twenty years ago (Smith et al, 1982). Here, the user has
two input devices: a keyboard and a mouse, and can use these to manipulate objects such as
documents, files and programs in the shape of icons and windows on a two dimensional
display. New technological advances now have made it possible to address the user in
multiple modalities and using 3D graphics in real time. With these new developments in
technology it is to be expected that new opportunities will arise, requiring new UI paradigms.
Although some UI developments have led to new types of applications, most of the newer
and more sophisticated user interfaces do not actually change the functionality of the system;
they do not add to what tasks the user can perform with the system. However, they do change
the usability of the system. Often seen as the main goal in HCI, the 'Holy Grail' so to speak,
the term usability is used to denote the ease with which the user can work with the system, or
to be more precise: the efficiency, effectiveness and satisfaction with which the user can
achieve his or her goals with the system (ISO 9241). Usability therefore deals not only with
the functionality provided by the system, but especially with the way in which the user can
access this functionality. The usability of a system is one of its main success factors (Nielsen,
1993). It should therefore be taken into consideration in the design of the system.
1.1 Virtual Reality
One new technology for which the WIMP interface is not applicable is Virtual Reality (VR).
The term VR was coined in 1989 by Jaron Lanier, although Sutherland (Sutherland, 1965;
1968) is usually considered to be the father of this new technology. The term is used to
indicate the use of computers, displays and sensors to create the illusion that the user is in
another environment than the real one, in a Virtual Environment (VE). VR makes use of real-
1

Chapter 1
time 3D computer generated graphics, which can be displayed on ordinary or stereoscopic
screens or inside a VR-helmet. Often, VR uses sensors to determine the position of the user’s
head to adjust the viewpoint accordingly, giving the user the impression that the VE
surrounds him or her.
VR systems are used for certain types of tasks for which traditional user interfaces are less
suited. Currently, VR is used amongst others in training and education, where users can learn
how to operate complicated machines such as airplanes or how to work in dangerous
environments such as a building on fire, in entertainment such as videogames, in
visualization, for instance allowing people to walk through buildings that haven't been build
yet (Brooks et al., 1992) and even in psychological therapy.
However, most of these applications of VR have not been in common use. One reason for
this could be the fact that until now, little research has been devoted to the user interface and
interaction techniques for immersive VR (Bowman, 1999). In other words, they lack
usability.
Why is it that VR systems are not that easy to use? VR, when it was first introduced, was
hailed as the ultimate display (Sutherland, 1965). Some argued that interaction in a VE
should be completely natural, because it replicates the way we interact with the real world
(Nielsen, 1993). However, for several applications, the way we interact in the real world is
not very suitable (Bowman, 1999), because we want to use the VR precisely because we
want to be able to do things not possible in the real world. Furthermore, interaction in VR is,
at the current state of technology, not at all similar to how we interact with the real world
(Kaur, 1998). Current limitations to this technology include the absence of tactile feedback,
limits of the tracking device forcing the user to use other, less natural, means to navigate
through a virtual world or limits in the realism of the behavior of the system, such as having
the ability to walk through walls. For desktop VR, the user is further limited because the
input devices are often not very suitable for 3D. Both mouse and joystick work in a 2D
plane, thus often imposing a limit on the degrees of freedom that a user has.
1.2 Presence
Since there are these difficulties in the HCI of VR systems, why should we even want to use
VR? When we look at the applications of VR as mentioned above, we see that they use VR
for a common purpose: to give the illusion of being in a different place than the real world;
of simulating another environment that is possibly similar to real environments, but has the
advantage of not being similar in certain respects such as the danger, costs or scale of the
environment that is simulated. This illusion of being in a different place is essential for the
effectiveness of these applications.
To denote this illusion, the term presence has been defined to describe the sense of being in a
virtual world, and has been the source of much research. Presence indicates the level to
which humans respond to a virtual environment as if it were real. Some authors see presence
as one of the defining aspects of VR (Steurer, 1992). Because of presence, users of an
architectural walkthrough will know what it is to be in the building before it is actually build
and can gamers experience the sense of being in fantastic worlds. The research into the sense
of presence highlights a shift of focus in HCI research that is already taking place for some
time. Previously, HCI focused mainly on the cognitive aspects of the user and the UI. Slowly
however we are becoming aware that computer systems can also have non-cognitive effects
on the user that should be taken into account when designing the UI. (Reeves & Nass, 1996)
showed that humans have a strong tendency to respond to computers in similar ways as they
2

Introduction
do to other humans. In presence we see humans responding to virtual scenes as if these were
real, at a non-cognitive level. We could say that the computer system no longer only supplies
the user with information; it also supplies the user with an experience (Laurel, 1993).
1.3 A new class of applications
The capability of VR to address the user at both the cognitive level and the non-cognitive
level brings great opportunities with it. We can create systems that offer users a designed
experience. However, it also introduces complexities into the design of such systems. To
understand these complexities, we will use the taxonomy proposed by (van der Mast, 1995).
This taxonomy states that the complexity of an application from the designer’s point of view
depends on the intertwining of three components: the user, the UI and the domain. This
taxonomy identifies at least four classes of applications, based on this intertwining:
One, non-reactive applications with functionally isolated domain and UI component, e.g. all
conventional administrative and financial applications with simple transactions and
command like user interfaces. The complexity is located mainly in the domain component.
Two, applications where the domain mostly or only deals with the UI, e.g. graphical editors.
The complexity is located in the UI component and in the required data structures.
The third class is that where there is poor isolation of UI and domain components and a very
complex UI component where the user tasks are strongly structures and organized, e.g.
process control systems with a graphical UI. The complexity is located in the UI, the domain
components and the intertwining.
The fourth class is an application that has the domain positioned in the interaction itself
where the user is, to a certain extent, considered to be part of the domain, e.g. software for
entertainment and education. The complexity is located and spread over all three
components, including the user and his or her behavior.
Most VR applications belong to the fourth class, where the user is influenced by the UI at
both the cognitive and non-cognitive level. Applications in this class are difficult to develop,
since it requires not only an understanding of the user, the UI and the domain, but also of the
effects that each component has on the other components.
Recently, we have seen even further complications in these types of applications; VR is
increasingly being used by multiple users simultaneously. A good example is the mounting
popularity of multi-player 3D games such as Unreal Tournament and Counterstrike, games
specifically designed to be used by more than one player. Furthermore, training applications
such as professional flight simulators often include both a trainee and an operator
simultaneously using the same system but using different means of interaction. We could
identify these as a new class of applications: Applications in which the system is used to
influence at least one of the users and where several users are working together in real time,
often using different types of user interfaces. Examples of applications in this class are the
aforementioned flight simulators, but also other training applications for instance for soldiers
in the battlefield or firefighters in hazardous situations. Another example will be discusses in
extensive detail later on: psychological therapy.
The complexity of these applications lies not only in the intertwining of the user, UI and
domain components, but also in the interactions between the various users: What are the
characteristics of the interaction between the user and the computer and of the collaboration
between different users in a cooperative VE, especially when they use different means to
interact with the VE? How can we design the computer system to best support this human-
3

Chapter 1
computer interaction and this collaboration between the users to optimize the usability? What
effect does this collaboration have on the sense of presence?
To begin answering these questions, we should start by selecting a single application that
embodies these new developments. This will enable us to ground our understanding in a real
world situation and it can be our test case to test that understanding. The application chosen
for this purpose in this dissertation is Virtual Reality Exposure Therapy (VRET).
1.4 Virtual Reality Exposure Therapy
In psychological therapy, we can use the fact that VR can affect the user at a non-cognitive
level to carefully administer synthetic experiences to the patient necessary to cure his or her
mental illness. Particularly in phobia treatment, VR has already been shown to be effective
with a great number of case studies and several controlled studies 1 . At this moment, phobia
treatment in VR can be said to be outgrowing the experimental stage and is already in use by
a handful of early adopters.
Phobias are characterized by a persistent fear of a
specific object or situation. Phobias are the most
common form of anxiety disorders, which themselves
are the most common psychiatric disorders. In 1998 a
study was done in the Netherlands on the prevalence of
psychiatric disorders in the general population (Bijl et
al., 1998). This study shows that the lifetimeprevalences
of agoraphobia 2 , social phobia and specific
phobias are respectively 3.4, 7.8 and 10.4 %. This
means that 3.4 % of the general population is or has
been suffering from agoraphobia during his or her life.
7.8 % and 10.4 % of the population is or has been
suffering from respectively a social phobia or a specific
phobia during his or her life.
Phobias can be treated by exposing patients to anxietyproducing
stimuli while allowing the anxiety to
attenuate (Bouman et al., 1992). Traditionally, these
stimuli have been generated by presenting the patient
with actual physical situations (In Vivo) or by having
the patient imagine the stimulus (In Vitro). Now, VR
offers a third option: Virtual Reality Exposure Therapy
(VRET). In VRET, the patient is confronted with a
fearful situation by displaying a VE with anxiety
provoking elements such as the VE depicted in figure
1.1 on a display such as a Head-Mounted Display
(HMD) as shown in figure 1.2. During the therapy, the
therapist in collaboration with the patient alters the
synthetic experience of the patient in such a way that
4
Figure 1.1: VE for treatment of
fear of heights (Courtesy of
Virtually Better Inc.)
Figure 1.2: Psychological
treatment in VR, with the patient
wearing a HMD (Courtesy of
David Ready Ph.D.)
1
For a brief overview of the history of phobia treatment in VR, see chapter 3 of this
dissertation.
2
Agoraphobia is the fear of being in places or situations from which escape might be
difficult or embarrassing. (American Psychiatric Association, 1994)

Introduction
the patient can slowly habituate to more and more fearful situations. For this, the system
provides two user interfaces: one for the patient and one for the therapist. The UI for the
patient incorporates advanced VR technology since the patient needs to be immersed in the
VE, the interface for the therapist is often more conventional. Patient and therapist need to
cooperate and coordinate their actions, requiring communication and interaction using these
different user interfaces. For systems that are currently in use, these interfaces are very crude
and simple, mainly dictated by the limitations of the hardware instead of the needs of the
user. As stated before, it is not yet apparent how to design such interfaces to provide for high
usability.
Furthermore it seems obvious that it is necessary for the patient to have a sense of presence,
feeling present in the anxiety-evoking situation to experience a similar emotional response as
in In Vivo therapy. Here we see the importance of presence for VRET: If users do not feel
present in the fearful situation then they will not experience fear and this fear will therefore
not diminish through habituation. We thus need a system that has the ability to produce a
sense of presence in the patient.
The sense of presence a person experiences in a VE depends on several factors, such as
display fidelity, the number of modalities stimulated (e.g. vision, sound, touch) and the
individual characteristics of the user. Furthermore, most scholars attribute a special role for
the interaction between human and computer in generating a sense of presence. However, the
exact role of this HCI in presence and thus in VRET is not yet clearly understood.
1.5 Research goal
In the previous paragraphs we have established that technological developments have
resulted in a new class of applications of this technology where users are exposed to
multimodal synthetic experiences that influence them at both the cognitive and non-cognitive
level. Often, they do not operate these systems alone but are collaborating with other users
who possibly have different means of interacting with the system.
The design of the user interfaces of such systems is a challenge. We currently lack a standard
paradigm for interactions with virtual worlds, we have no proven set of interaction
techniques that provides a high degree of usability and specifically enables the user to
experience a sense of presence. It is even unclear what changes should be made to the
design process to facilitate the design of these multi-modal interactions between user and
computer and also between the different users.
In this thesis we will make a start at meeting the challenge of designing the user interface for
such systems. In order to do this, we will focus on a case that is representative for these
applications: VRET. By undertaking the task of designing the user interfaces for this
application we should not only gain insight into the design process required for such
applications. We should also be able to contribute to the construction of a new UI paradigm
that is suitable for these new technologies, similar to the WIMP paradigm for 2D interfaces.
This can be achieved by determining what type of interaction techniques and metaphors offer
the highest usability, in terms of the ease with which user can work with the system, and the
highest effectiveness in terms of the effect the system has on the user at the non-cognitive
level.
We will therefore formulate the research goal as follows:
5

Chapter 1
Propose and validate guidelines for the Human-Computer Interaction of Virtual Reality
Exposure Therapy Systems for treatment of phobias to improve the usability and
effectiveness of these systems.
The term HCI in the broadest sense includes all aspects related to the interaction between
human and computer, ranging from the hardware technology to the layout of the computer
screen. However, to limit the scope of our research, we will use a more narrow definition of
HCI, incorporating only the dynamic aspects of HCI. This includes only those aspects that
change as a consequence of human interaction. Therefore, we will not look at for instance the
fidelity of displays nor at the number of modalities stimulated by the displays, since these
aspects do not change as a consequence of user interaction.
1.6 Dissertation outline
In this chapter, we have introduced the research and research goal we will address in this
dissertation.
In chapter 2 the research approach will be outlined that states, in brief, that before we can
design the HCI for VRET, we will need a proper way to analyze the way the therapist and
patient work, and a more thorough understanding of the way HCI effects presence and the
effectiveness of the therapy.
In chapter 3, the history of VRET is reviewed, as well as the task analysis methods currently
available to analyze the VRET process. An adaptation of current task analysis methods that
will enable us to analyze the VRET process is described and applied to the case of VRET,
resulting in a task model and an understanding of the usability problems facing VRET
systems today.
In chapter 4 the research on presence will be reviewed, resulting in a model of the way HCI
might influence presence and therapy effectiveness.
The task analysis and presence model will be used to generate requirements for the different
user interfaces of the VRET system and to generate new design proposals. These
requirements, proposals and their evaluation will be described in chapters 5 and 6, dealing
with the patient’s and therapist’s UI respectively.
In chapter 7 we will discuss the consequences of the results of these design proposal
evaluations for the HCI of VRET and for our task analysis method and presence model. We
will close this chapter and this dissertation with several general conclusions.
6

2 Research approach
Computer science research studies artificial as opposed to natural phenomena. It deals with
human creations and these human artifacts can be both created and studied (March & Smith,
1995). Thus, both natural sciences and the design sciences can be applied to information
technology. The natural sciences are concerned with explaining how and why things are,
design sciences are concerned with devising artifacts to attain goals. The research described
in this dissertation is mainly concerned with the devising of artifacts, requiring a design
methodology. This methodology consists of mainly two activities: building and evaluating.
The first question in determining our research approach then is: what should we build? The
products one can build in the design sciences are of four types (March & Smith, 1995):
constructs, models, methods and implementations. Constructs or concepts form the
vocabulary of a domain, a model is a set of propositions or statements expressing the
relationships among constructs, a method is a set of steps used to perform a task and an
implementation or instantiation is the realization of an artifact in its environment.
Instantiations operationalise constructs, models and methods and can demonstrate the
feasibility and effectiveness of the methods and models they contain. In this research, the
products that we need to build are those that are necessary to facilitate the design of the user
interfaces of the class of applications of which VRET is our prime example. Those products
are:
1. A means of dealing with the question how to design such systems: a design method.
2. An understanding of the relationship between the HCI, presence, fear and effectiveness
of the therapy: a model describing the relationships between these constructs.
3. An instantiation incorporating the former two: the design of the interfaces for both the
patient and the therapist for VRET.
Whereas natural science tries to understand reality, design sciences attempt to create things
that serve human purposes. Its products are therefore assessed against criteria of value or
utility. The second question we need to answer to complete our research approach is: what
criteria should be used to evaluate our products? The basic evaluation criteria are: (March &
Smith, 1995; Dolby, 1996)
• Does it work?
• Is it an improvement? (if relevant)
A third criterium can be added: the generalizability: does the product maintain its value
when placed in another environment? This criterium is especially relevant in this dissertation
since our case of VRET is intended to be, to some extent, representative for the applications
in its class. Will the findings based on this case hold for other similar applications as well?
The building and evaluating of the various products will take different approaches and will
be discussed separately:
2.1 Design method
A design method sets forth how one should go about designing an interactive system. Two
domains can be distinguished in developing such a system: behavioral and constructional
(Gabbard et al., 1999). The behavioral domain represents the view of the user and user
7

Chapter 2
interaction with the system, while the constructional domain represents the view of the
system developer and the overall system. The latter domain has been the subject of
traditional information system development methods. In contrast, developing the user
interaction component is part of the behavioral domain. To create a system that is usable,
usability has to be taken into account in the development process. This requires an altogether
different design method. Already several of such HCI design methods are available and these
will be reviewed in the next paragraphs.
2.1.1 Existing HCI design methods
Generic HCI development methods
In the usability engineering lifecycle as described by Nielsen
(Nielsen, 1993) we find a strong emphasis on focusing on the user
and his/her task. The various ways in which Nielsen proposes we do
this is by involving the user, also at the earlier stages of development,
and by thoroughly analyzing the user and the user’s task. The
analysis of the user’s task is referred to as a Task Analysis, and for
several scholars (e.g. Sebillotte, 1995; van der Veer & van Welie,
1999) it is basis of HCI design. In general, the HCI design process
starts of with such an analysis of the user’s current way of working.
Based on any usability problems found, a second model is created,
this time of the desired way of working, and a design of the system
that should support and enable this new way of working. This new
design is then evaluated, often through heavy involvement of the
users, and possibly redesigned. The generic design process is
depicted in figure 2.1.
Task
Analysis
Usability
problems
Design
alternatives
Design
evaluations
Figure 2.1: Main
steps in the generic
HCI development
process
VE development process
VR-systems are a unique class of systems. (Johnson, 1998) warns that traditional HCI
development techniques might be inappropriate for VR-systems because first of all it can be
difficult to define the future user population since most VR systems are build for a very
general audience. Second, it can be difficult to define meaningful tasks for VR-systems
because the majority of VR applications are intended to be used for instance for “learning
through navigation”, where the user is required to gain an understanding of the VE by
navigating through the space, something which is common for training applications. In this
case, there is no procedural task the user will follow which can be supported by the user
interface of the system. Task analysis in VR should therefore be investigated to understand
how these aspects can be incorporated.
Apart from this it is likely that iterative development will play a large role in VR
development because “unlike most current human-computer interfaces, there are rather
severe design trade-offs, e.g. between responsiveness, world complexity, and the quality of
images and sounds rendered.” (Stuart, 1996) Furthermore, because of the novelty of VR,
many important aspects in VR development are still unknown and can only be found through
trail and error.
An investigation of current design practice for VEs (Kaur, 1998) has shown that design is
often informal and that an iterative approach is used, partly because most VR-development
tools allow quick adjustments of the VE followed by subsequent viewing of the VE by the
8

Research approach
developer. However, very little user testing is done. In general, the steps taken in the
development process are:
1. Requirements specifications
2. Gathering of reference materials from real world models
3. Structuring of graphical models, sometimes dividing it between designers
4. Building objects and positioning them in the VE
5. Enhancing the environment with texture, lighting, sound and interaction, and optimizing
the environment
Most time is spend on creating the appearance of the VE, with little effort spend on usability
issues. Important HCI issues such as interaction techniques and input devices are not dealt
with by current VR application developers.
2.1.2 VRET Design method
In the previous paragraphs we have reviewed the steps deemed necessary to develop a
system that is usable. As can be expected, the user plays a key role in the process, and close
collaboration between developer and user is essential. However, in practice, the development
of systems incorporating VR is not done according to these guidelines, and also very little
research has been performed investigating how usability can be improved in of VR systems.
A problem identified in designing VR systems is the analysis of the user’s tasks. For VRET
the TA will be complicated even further because of the intricacies of the communication
between the different users. Hence, our primary focus in adapting current design methods for
VRET will be on improving the TA method to facilitate the unique characteristics of VRET
applications. This will be discussed in detail in chapter 3.
2.1.3 Evaluation
The first test of our design method will be to see if it works. The fact that our method works
can be shown most easily by actually applying this method in designing the user interfaces
for VRET, which is what we will do in this dissertation. As such, the design method is
simultaneously an integral part of the research approach. By showing that the instantiation of
our method, i.e. the UI for both the patient and the therapist are usable we have shown that
this method can be used.
However, since there are already a number of other methods for designing interactive
systems and especially for TA, we will also have to show that the method is an improvement.
This is far more complicated and, when done correctly, should involve the comparison of the
usability of this method to other methods. A real comparison, where several such methods
are applied to several cases and their use evaluated on the use of the method and the quality
of its products, is both extremely resource-consuming and almost impossible, since the
outcome of the application of a design method depends on a great number of variables that
cannot be known beforehand and can therefore influence the results of such a hypothetical
experiment. When proposing a new design method, it is therefore customary in computer
science to show that the method is an improvement by arguing for its advantages over other
methods and making it plausible that it is an improvement. Since our most important
contribution to the design method will be the TA method, we can evaluate this method in
terms of the validity of the models that are created using this method (i.e. their conformity to
reality) and their usability in the subsequent designing of a new UI.
9

Chapter 2
In a similar vein, the generalizability of a method is hard to prove outright and is also done
by argument alone.
2.2 Presence model
In order to design a system that is effective, we will need an understanding of the way in
which the HCI affects the effectiveness of the therapy. The simplest way to do this would be
to manipulate one HCI variable at a time and observe its effect in real therapy. However, this
would be extremely time-consuming and require many therapists and patients before
significant results are obtained. We therefore need to divide the problem into smaller
problems, using intermediary concepts such as presence and fear. This way we can determine
the overall effect of HCI on effectiveness by determining the relationships between these
intermediary concepts, which are much easier to investigate and at the same time provide us
with a more thorough understanding of the underlying process. We can furthermore make
use of the considerable amount of research that has already been performed in this area.
Review of this research and the formulation of such a model are described in detail in
chapter 4.
2.2.1 Evaluation
Since this will be the first of such a model attempting to explain the effectiveness of therapy
in terms of the HCI, it is irrelevant to ask whether it is an improvement. We will only have to
show that it works. This can be tested by showing that predictions made using this model
about the UI for VRET are correct. Here, the hypothetico-deductive method is appropriate:
we can deduce hypotheses from the model and test these hypotheses using empirical data.
These hypotheses should apply to the case of VRET.
The HCI in VR includes navigation and object selection and manipulation. The interaction
most generic for all phobias and other applications than VRET as well is navigation. We will
therefore focus on the patient navigation, which can be divided into wayfinding and
locomotion. Wayfinding is the cognitive process of determining a path, locomotion is the
control of the viewpoint. In this dissertation, wayfinding techniques will not be discussed,
since the virtual environments used are relatively simple, and we assume that wayfinding has
no therapeutic value. Thus, the key HCI issues concerning the patient interaction in VRET
are related to patient locomotion: should the therapist or the patient have control over the
locomotion of the patient through the VE? If the patient is allowed control over the
locomotion, which interaction technique provides the highest effectiveness?
The presence model that will be presented in chapter 4 suggests that presence is essential to
provide the patient with fear, and that fear is essential for effective therapy. We can test the
effect of the different choices related to patient locomotion with regard to presence and fear.
First of all, we can test the effect of the choice of patient control versus therapist control:
Hypothesis 1: Locomotion controlled by the patient will increase the patient’s sense of
presence.
Hypothesis 2: Locomotion controlled by the patient will increase the fear a phobic user can
experience.
10

Research approach
Second, we can test whether, in the case of patient control, there are any differences between
the various locomotion techniques. For presence and fear, it seems most likely that a more
natural locomotion technique will lead to higher presence and thus to more fear. A more
natural locomotion technique is one with which the user is closely familiar. Therefore,
different locomotion techniques will be natural to different to users. However, the
locomotion technique most common to all users is that which we use in everyday reality:
walking with our own two feet and looking around by turning our head and body.
Taking this notion of natural in consideration, we can formulate the hypotheses:
Hypothesis 3: A more natural locomotion technique for the patient will increase the presence
a user experiences
Hypothesis 4: A more natural locomotion technique for the patient will increase the fear a
phobic user can experience.
2.3 Patient and therapist UI
The patient and therapist each have their own UI to interact with the system. The designs for
the patient and therapist UI describe what the HCI should look like for both users, the
functionality offered by the system and the way in which the user can access this
functionality. They can said to be instantiations of the design method combined with the
presence model since they will be built using these two research products.
2.3.1 Evaluation
To determine whether these user interfaces work, we will need to understand their
consequences for the usability and effectiveness of the system.
The effectiveness of the system could be evaluated by determining the effectiveness of the
entire treatment using the VR system, but as stated in the previous paragraphs on the
presence model, this would be extremely complicated and time consuming. We will
therefore use intermediary concepts such as presence and fear, and evaluate whether the user
interfaces are effective in generating a sense of presence and adequate fear levels in the
patients. We then need to show that presence and fear contribute to higher treatment
effectiveness.
To evaluate the usability of the user interfaces, we need to go into the domain of software
usability as already introduced in the previous chapter. There, we used the ISO definition of
usability (ISO 9241): The effectiveness, efficiency, and satisfaction with which specified
users achieve specified goals in particular environments. Effectiveness is defined as the
accuracy and completeness with which users achieve specified tasks and efficiency is the
resources expended in relation to the accuracy and completeness with which users achieve
goals. Satisfaction is a subjective measure and concerns the comfort and acceptability of use
by end users.
(Nielsen, 1993) uses a definition that is specified in elements that are more specific, as
shown in table 2.1.
11

Chapter 2
ISO 9241 Nielsen
Efficiency
Efficiency
Learnability
Memorability
Effectiveness Errors/Safety
Satisfaction Satisfaction
Table 2.1: Usability as in (ISO 9241) and (Nielsen, 1993), adapted from (van Welie, 2001)
The usability aspects defined by Nielsen provide a practical operationalisation that we will
use in this dissertation. In literature, several methods have been proposed for determining the
usability of a system, which will be reviewed in the next paragraph.
2.3.2 Determining usability
A wide range of different methods for performing usability evaluations have been proposed,
each with its own body of advocates (Holyer, 1993). Unfortunately, even though attempts
have been made to compare these methods, most of these are hampered by inadequacies in
the design of these studies (Gray & Salzman, 1998).
Usability evaluation can take place by investigating conformity of the design to heuristics
and guidelines. Another possibility is to test one or several prototypes with real users.
Usually, several evaluation methods are combined. For instance, Hix et al, when developing
a VE for battlefield visualization, used heuristic evaluation followed by user evaluation of a
single prototype, and summative evaluation, where users are required to compare two
designs (Hix et al., 1999). The main reason for first performing heuristic evaluation is that
this type of evaluation is relatively cheap; no prototype had to be constructed.
Heuristic evaluation
Heuristic or expert evaluations are performed by usability experts who check the design for
possible usability problems. Guidelines often form the basis of such an evaluation, and these
will be discussed in more detail in the next paragraph.
Another method for heuristic evaluation is the use of a cognitive walkthrough. Here, the
usability experts will go through the design following a scenario of a user working with the
system, often depending on the expertise of the evaluator to identify possible usability
problems. This process can even be automated using a Programmable User Model (Amant,
1999) that evaluates whether for each task there is a chain of user interface commands that
will lead to the successful completion of that task.
The proposed task situation can be modeled in more detail using a TA method. This model
can then be used to make predictions about the usability and the performance of the design.
Several methods apply techniques reminiscent of the time studies used in the Scientific
Management method (Taylor, 1911), extending these to cognitive labor as well as physical
labor. For instance the GOMS method includes a Keystroke-Level model (Card et al., 1983)
that can be used to predict the time required to complete a certain task.
Guidelines and principles
Guidelines and principles can help the evaluator in applying existing usability knowledge to
the design. Principles are high-level usability requirements and are too general to be applied
12

Research approach
directly, so they have to be translated into design guidelines. Guidelines can be applied
directly during the design phase as well as during an evaluation.
Several extensive sets of guidelines have been developed for traditional user interfaces, e.g.
(Apple, 1987) (ISO 9241, 1997). Some work has been done in creating guidelines
specifically for VR-systems. Gabbard and Hix determined a large number of general
guidelines based on literature studies and interviews with researchers (Gabbards & Hix,
1997). (Kaur, 1998) developed a number of guidelines based on models of human interaction
cycli during interaction with VEs supported by user evaluations of several non-immersive
virtual environments.
Guidelines are often very general, making it difficult to interpret them in a specific context.
A solution could be to generate guidelines that are context and artifact specific (Sutcliffe &
Caroll, 1999). This will reduce the applicability of the guidelines in other domains but will
increase the effectiveness of applying the guideline within the specific context and domain.
User evaluation
User evaluation requires users to test a prototype of the system. The users can be asked to
simply explore the system, or can be given specific tasks to complete. This task can be
representative of the type of work that the system will eventually be used for, it can also be a
task aimed at specifically testing a part of the user interface. For instance a standard battery
of tasks for VEs can be used, including tasks such as navigating in a corridor or reading a
virtual eye chart (Lampton et al., 1994).
Information that can be gathered from the user during the test includes performance
indicators such as completion time and number and severity of errors, as well as more
subjective information (Kalawsky et al., 1999).
Subjective information can be acquired using for instance the think-aloud protocol described
in the paragraph on data gathering techniques in chapter 3. Kaur (Kaur, 1998) used this
method to identify usability problems in several desktop-VR applications.
A variant of the think-aloud protocol is the co-operative evaluation, a method requiring two
users to use the system together, which is shown to be effective in evaluating a desktop VR
system (Marsh & Wright, 1999).
Another way to acquire subjective information on the usability of a system is through use of
a questionnaire. Questionnaires, such as the VRUSE (Kalawsky, 1998), have been developed
specifically for VR-systems.
Instead of testing a single design, several alternatives can be compared through user testing,
a process known as summative evaluation.
2.3.3 Determining usability in VRET
Heuristic evaluation and the use of guidelines and principles require a thorough
understanding of the UI and the type of user interfaces. Unfortunately, for VR very little is
yet known about the UI and its effect on the user. Furthermore, we cannot use our own TA
method to evaluate the design proposals based on that method, since this would be begging
the question. We therefore must rely on more empirical evaluation methods, incorporating
real users, in order to establish the usability of the different design options. As discussed
before, user evaluation can be performed by gathering subjective information from the users
themselves through use of for instance questionnaires or the think-aloud protocol, or by
gathering more objective information about the performance of the user on specific tasks.
When considering the results of the investigations of this dissertation, one must however
13

Chapter 2
keep in mind that subjective evaluations are, by their very nature, more prone to errors and
misunderstandings, reducing the reliability of some of the findings.
2.3.4 Evaluation of the patient UI
The usability of the UI for the patient is of relative little importance when compared to the
effectiveness of this interface in treating the patient. Evaluation of this effectiveness has
already been discussed previously with relation to the evaluation of the presence model. The
usability for the patient should however not be ignored completely and during the
experiments described in this thesis the ease with which patients can achieve goals during
therapy can be measured by recording the accuracy with which they can position themselves
in the VE and their subjective evaluation of the UI, and especially the locomotion technique.
2.3.5 Evaluation of the therapist UI
First of all, we can expect locomotion by the patient to also have an effect on the HCI of the
therapist. We believe that patient control over the locomotion will relieve the therapist of the
task of moving the patient through the VE and will thus reduce the complexity of the HCI for
the therapist. However, this is not completely obvious since it will require more cooperation
and communication for the therapist to get the patient to move to a specific location. We will
formulate the fifth hypothesis as follows:
Hypothesis 5: Locomotion controlled by the patient will reduce the complexity of the HCI for
the therapist.
By far the easiest way to test this hypothesis is by simply asking the users for their subjective
evaluation in a comparative evaluation of the two types of control: patient control and
therapist control over the patient’s position. For this two evaluation tools are most feasible:
an evaluation similar to the think-aloud protocol and post-test questionnaires. Unfortunately,
as will be explained in more detail in chapter 3, the think-aloud protocol is note very suitable
in the case of VRET, since it will disturb patient-therapist communication. We will therefore
rely on post-test user comments when reviewing video recordings of those users at work with
the system and responeses to questionnaires.
Current user interfaces of VRET systems do not have a screen specifically designed for the
therapist but show the same image to the therapist as displayed in the HMD of the patient.
This setup severely limits the flow of information to the therapist. To solve the usability
problems relating to this lack of information, such as problems in navigating the patient and
dependence on elaborate instructions to operate the equipment, we propose to provide the
therapist with a graphical user interface (GUI) especially designed for the therapist. This
GUI could then also assist the therapist in determining the fear level of the patient. This GUI
will incorporate both 3D elements, displaying the VE from various viewpoints, and 2D
elements similar to current GUIs. We expect that such an interface will increase the usability
of the system.
Hypothesis 6: An extended therapist UI in the shape of a combined 2D – 3D console will
increase the usability of the system.
Similar to the previous hypothesis, a comparative evaluation seems appropriate, this time
comparing the current user interfaces without the extended console to a newly designed
14

Research approach
interface that does provide the therapist with a combined 2D – 3D console. Again, post-test
questionnaires and therapist comments during video review will be used to measure the users
subjective evaluation of each of usability of the two alternatives.
This hypothesis can be specified to describe the various elements that we propose to
incorporate in this interface.
First of all, we can include a tool specifically designed to aid the therapist in keeping track of
the patient’s fear. Therapists often request patients to report their fear in Subjective Units of
Discomfort (SUD), a scale from one to ten. Our tool could record these SUDs, thus helping
the therapist in gaining an understanding of the changes in the fear of the patient over time.
Hypothesis 6a: Providing a tool for keeping track of the patient’s fear will increase usability
for the therapist.
To solve the problems related to moving the patient through the VE we propose to add a
second 'external' viewpoint with a projection of real world objects and an autopilot based on
a 2D map.
Hypothesis 6b: Providing the therapist with an ‘external’ viewpoint of the VE with a
projection of real world objects will increase usability for the therapist.
Hypothesis 6c: Providing the therapist with an autopilot based on a 2D map will increase
usability for the therapist.
We can expect that especially the external viewpoint of the VE might be able to reduce
navigation errors by the therapist and reduce the time to complete tasks in the VE. It is
possible to test this in a more objective way than is possible for the other elements of the
combined 2D/3D elements. Specifically for the external viewpoint, we can therefore perform
a comparative evaluation to determine whether the addition of this viewpoint reduces errors
and increases speed when performing the task of navigating a patient through a VE.
Finally, we can also include standard graphical control widgets in our GUI, by which the
therapist can operate the VE. This will provide the therapist with affordances regarding the
possible options and how to enable them, thus reducing the need for elaborate instructions
and manuals.
Hypothesis 6d: Providing the therapist with graphical controls over specific elements in the
VE will increase the usability for the therapist.
The testing of the hypothesis will be described in chapter 5 and 6, which deal with the
patient’s and therapist’s UI respectively.
2.3.6 Further evaluation
Since there already are systems available for treatment of phobias, we will need to show that
the user interfaces of our system are an improvement over these other systems. To do this we
will create an inventory of the characteristics of the user interfaces of these systems and
determine the current state-of-the-art. Comparing our designs with this state-of-the-art will
have to show that we have made an improvement.
15

Chapter 2
The designs for the patient and therapist UI are the products of the design method combined
with the presence model. As such they are valuable in evaluating these research products as
discussed in the previous paragraphs. Needless to say, they are also valuable since they are
prescriptive of what user interfaces should look like for VRET applications. Furthermore,
some aspects of these user interfaces are generalizable to other types of applications within
the same class as VRET as well. For a few aspects, such as the locomotion technique for the
patient, we might even expect that these will contribute to a golden standard for what a UI
for VR technology should look like, similar to the WIMP interface for 2D graphical user
interfaces. However, proving this generalizability falls beyond the scope of this dissertation
and will only be looked into briefly.
2.4 Conclusions
In this chapter we have discussed the research approach necessary to fulfill our research goal.
Since the research goal entails the design of artifacts, we have stated that a design
methodology is appropriate. The artifacts that need to be designed are: a design method, a
model describing the relationship between the HCI and the effectiveness of VRET therapy
using the concept of presence, and the designs of the user interfaces for our system that are a
product of the former two artifacts.
These artifacts will have to be built and will have to be evaluated. Building the design
method and presence model will be described in chapter 3 and 4 respectively. The design of
the user interfaces will proceed according to these products and will form the basis for
evaluating them. In chapter 3, our TA method will be applied to the case of VRET. The
evaluation of the designs resulting from this analysis and the presence model will be
discussed in chapter 5 and 6.
16

3 Analysis
As stated in the previous chapter our primary focus when creating a new design method
specifically tailored for developing applications such as VRET is on the Task Analysis (TA)
phase of the design process. According to most researchers, the TA is the basis for designing
interactive systems. It is used not only for identifying usability problems but also forms the
basis for future HCI designs.
We will begin this chapter by taking a closer look at our application of choice: VRET. We
will do this by reviewing the brief history of VRET. After this, we will look at existing TA
methods and the extent to which they can be applied to applications such as VRET. Several
adaptations will be proposed to improve the existing TA methods.
By subsequently applying our adapted TA method to VRET, we can both demonstrate how
the TA can be applied and evaluate its usefulness. For this, we will need to look at the
possible sources of data concerning VRET and the techniques that can be used for gathering
this data. Gathering and analyzing the data using our TA will result in a Task Model (TM) of
the HCI in VRET and insight into the areas in which the HCI could be improved.
3.1 A Brief History of VR Exposure Therapy
Phobias are a common mental disorder and can be divided into three main types (American
Psychiatric Association, 1994):
• Agoraphobia: The fear of being in places or situations from which escape might be
difficult or embarrassing.
• Social phobia: The fear of situations in which the person is exposed to possible scrutiny
by others and fears that he or she may do something or act in way that will be
humiliating or embarrassing.
• Specific phobia: The fear of a circumscribed stimulus (object or situation), such as the
fear of heights or the fear of enclosed spaces.
Researchers have already experimented with treatments of a wide variety of these phobias
using VR. In the following paragraph we will review these experiments to gain insight into
the current status of VR phobia treatment.
3.1.1 Fear of Heights
The first studies concerning treatment of phobias using VR were conducted at the end of
1993 (Rothbaum et al., 1995) as a collaborative effort by computer scientists and therapists
from three Atlanta universities: Georgia Tech, Clark Atlanta and Emory. The phobia selected
was acrophobia: the fear of heights. Seventeen students, selected for their fear of heights,
participated in this study. Subjects were randomly assigned to either a treatment group or a
wait-list control group. The treatment group had one pre-treatment session to learn how to
work with the VR equipment. After this they participated in seven weekly 35- to 45-minute
sessions during which they were exposed to three virtual height situations: an elevator, a
series of bridges and a series of balconies. Anxiety, avoidance, distress, and negative attitude
towards height decreased significantly for the treatment group but not for the wait-list group.
Seven of the ten persons to complete the VR treatment exposed themselves to height
17

Chapter 3
situations in real life during the treatment period although they were not specifically
instructed to do so.
Clark University as well as Georgia Tech in collaboration with Emory performed follow-up
case studies that confirmed that the fear of heights reported by patients was reduced after VR
exposure therapy (North et al., 1996b).
Almost simultaneously with the acrophobia studies in Atlanta a physician at Kaiser
Permanente was doing similar work in California in collaboration with the VR Company
Division Inc (Lamson, 1994). Here, 40 subjects were exposed to one 50-minute session
where they completed a chosen task such as crossing the Golden Gate Bridge or riding in a
glass-enclosed elevator. According to the researchers, after this session, all of the subjects
were able to complete a task of riding up and down a glass elevator while looking outside.
At the Delft University of Technology, in close collaboration with the University of
Amsterdam, research of treatment of fear of heights in VR was continued, starting with a
pilot study (Schuemie et al, 2000), after which the research described in part in this
dissertation was performed.
3.1.2 Fear of Flying
Both Georgia Tech and Clark Atlanta followed up their acrophobia research with
independent fear- of flying studies. Georgia Tech conducted a case study (Rothbaum et al.,
1996) using a virtual plane which simulated a passenger seated by the window in a standard
commercial jet. The passenger could look out the window and see the ground and changing
sky scenes. The patient, a 42-year-old female with a debilitating fear and avoidance of flying,
was taught traditional anxiety management techniques. After this she received six twiceweekly
sessions of approximately 35-45 minutes. During each virtual exposure the subject
was allowed to progress at her own pace along a natural progression of scenarios: sitting in
the plane with engines off, sitting in the plane with the engines on, taxiing the runway,
smooth takeoff, smooth flight, close pass over the airport, landing, rough takeoff, turbulent
flight, and rough landing. The patient's fear of flying reduced significantly, although the
contribution of VR graded exposure cannot be determined given the prior inclusion of the
anxiety management techniques.
At Clark Atlanta two independent case studies for fear of flying were performed (North et
al., 1996a). The VR scenes simulated an Apache army helicopter departing and then flying
over an airport and a city scene. A virtual therapist, with whom the patient could
communicate during the trials, rode in a seat next to the patient in the computer-generated
scenes. Sound and vibration were included for realism. The patients were left in an anxietyprovoking
scene until they reported an anxiety level of zero. After respectively five and eight
exposure sessions, both patients reported a significant reduction in fear of flying.
Recently a controlled study was performed comparing VRET to both exposure treatment in
vivo and a waitlist, with 15 subjects in each of the three conditions. (Rothbaum et al., 2000)
All subjects in the treatment conditions first received 4 sessions of anxiety management
training after which VR treatment consisted of exposure to the VE used in the experiment
described above. In vivo exposure consisted of visiting an airport and an airplane on the
ground. Results showed no significant difference in effectiveness between the two treatment
groups, and a significant difference between treatment groups and the waitlist.
18

Analysis
3.1.3 Agoraphobia
In an experiment, 60 subjects suffering from agoraphobia were exposed to a variety of
scenes: A series of balconies, an empty room, a dark barn, a covered bridge, an elevator, a
canyon with series of bridges and balloons. When the research, which lasted a year, was
completed in February 1995 a significant decrease in patient anxiety was found. (Coble et al.,
1995)
3.1.4 Arachnophobia
Researchers at the Human Interface Technology Lab at
the University of Washington performed a case study of
treatment of arachnophobia, the fear of spiders (Carlin
et al, 1997). The subject was a 37-year old single
woman who displayed severe fear of spiders, even
resulting in extensive obsessive-compulsive behavior.
Over a period of three months she participated in twelve
weekly sessions approximately 50 minutes long. After
one month the visual stimuli was enhanced with tactile
augmentation; an object resembling a spider was
attached to a tracking sensor, its position corresponding
to a virtual object: the spider. The patient could then
touch the spider to enhance realism. At some time
during therapy the patient exposed herself to real
Figure 3.1: Virtual world used in
the treatment of arachnophobia
(Courtesy of Hunter Hoffman,
University of Washington)
phobic stimuli without being specifically instructed to do so, similar to certain patients in the
acrophobia-studies. The patient’s rating of fear of spiders, as measured by self-report and a
survey, decreased to the point where she was able to resume outdoor camping activities.
3.1.5 Fear of public speaking
Frequently identified among the top most prevalent phobias, fear of public speaking was the
subject of another study at Clark Atlanta University (Hodges et al, 1997). Comparing control
and test groups of undergraduates, subjects were placed in front of a virtual auditorium that
gradually filled with virtual people. Simulation of room and crowd noise included laughter,
commentary, and applause. The treatment schedule consisted of eight, 10- to 15-minute
weekly sessions. Two assessment measurements, the Attitude Towards Public Speaking and
SUDs (Subjective Units of Discomfort) scale showed significant reduction of anxiety
symptoms after virtual therapy treatment.
3.1.6 Claustrophobia
At the University of Valencia in Spain a case study was performed (Alcaniz et al, 1998)
treating claustrophobia: the fear of confined spaces. The subject, a 43-year-old widow, was
referred by the Mental Health Services because of her strong fear, anxiety and inability to
undergo a necessary CTS (Computed Tomography Scan). A CTS requires a patient to remain
in a confined space for some time.
Great detail was put into the virtual environment, consisting of three adjacent rooms: a
balcony or small garden of two by five meter, a four by five meter room with doors and a big
window which communicated with the balcony and a three by three meter room which
communicated with the previous one. This last room did not have any furniture or windows
19

Chapter 3
and the ceiling and the floor were darker, with wooden textures to give a sensation of greater
enclosure. Once the door was closed, the patient could lock it. There was also a wall that
could move at the patient’s will making a loud noise with four possibilities for advancing
toward the patient leaving her locked up in a space of one square meter. Patient anxiety and
avoidance-behavior was measured using several scales, each of which decreased during
therapy and were maintained at follow-up. Interestingly, the patient reported several times
that “the wall produces even more fear in me than the CTS does”, suggesting that virtual
experiences can be made to be even more frightening than real ones.
3.1.7 Conclusions
At this moment VR exposure therapy can still be said to be in an experimental phase.
Research, mostly in the form of case studies, has been focused solely on the effectiveness of
VRET. In the development of VRET, it has become clear that it can be regarded as a
substitute for exposure in vivo. Patients are exposed to the stimuli they fear and this helps in
reducing the phobia.
Currently, VR is used solely during the exposure itself. For phobias, no research has yet been
performed into using VR as for instance a diagnosis tool, in contrast with other mental
disorders such as ADHD (Rizzo et al., 2000).
3.2 Current Task Analysis methods
When creating a model of a situation for the sake of system design, the analysis is usually
performed using a pre-described method. This gives some assurance that all relevant aspects
will be captured, and that the result of the analysis, the model, does not include too much
irrelevant information.
In the early days, modeling in system design was focused on describing the way the system
20
Level 1
Level 2
Level 3
Pragmatics
Semantics
Knowledge
User(s)
Computer(s)
Cognitions
Syntax
Lexicon
User Interface
Goals
Tasks Domain
Media
Objects
Objects
Figure 3.2: Overview of the Task Analysis domain

Analysis
works (Benyon & Imaz, 1999). With the introduction of HCI issues, it became more
important to model the way the user works and what role the system can have. This has
resulted in a wide variety of modeling techniques, as illustrated in (Kirwan & Ainsworth,
1992) and (Scheffer, 1995). Even though most of these techniques are placed in the category
of Task Analysis (TA), there are a great number of differences between the individual
techniques (Whitefield & Hill, 1994). In figure 3.2 the potential domain of TA methods is
depicted, consisting of three levels of detail. On level 1, only the tasks are recognized and the
domain on which these tasks may have an effect. Most TA methods however also include
details of level 2 3 , distinguishing between users, possibly between computer systems and
different objects in the domain that are in some way involved in the tasks under analysis.
There also is a wide variety of TA methods that deal with specific details of one of these
components, such as the user’s cognitions or the pragmatics, semantics, syntax or lexicon of
the communication between user and computer or even between different users, and the
media through which this communication takes place.
In the following paragraphs an overview is presented of the most popular TA methods.
3.2.1 Modeling tasks
A model of the task performed by the user can aid the designer in capturing system
requirements and in problem identification where some system failure has occurred.
According to (van der Veer & van Welie, 1999), two types of TM can be distinguished:
• Task Model 1 (TM1): model of the current task situation
• Task Model 2 (TM2): model of the future task situation
The current task situation is the way the task is performed at this moment, before any
improvements to the system have been made. Modeling the current situation will enable the
developer to better understand the requirements of the task at hand and determine
shortcomings in the current way of working. The model of the future situation describes the
way the task will be performed after a proposed new design has been implemented, and will
highlight how redesigning the system will change the task.
However, others claim that there should be only a single TM, applicable to both current and
future task situations. One of the dangers in using a single TM for system design is that this
can lead to non-creative redesign of existing artifacts (Benyon, 1992). To avoid this problem,
the TM should then limit itself to modeling the user’s high-level goals, which are
independent of the existing task implementation. One way to ensure that the model only
contains elements at this abstraction level is using Sub-Goal Templates, which define a
specific boundary for the level of detail. (Richardson et al., 1998)
A TM can not only be used to describe an existing or designed situation; it can also be used
to evaluate a proposed way of working. For instance, the Goals, Operator, Method and
Selection rules (GOMS) method (Card et al., 1983) includes a keystroke-level model for
estimating the time required to perform a certain action using a proposed design.
In general, the domain of these types of TA is limited to level 1 and 2 of figure 3.2. It
describes the tasks, consisting of behaviors and structures to support those behaviors and
includes hardware, software and users. The tasks are performed within a work domain. The
work domain contains physical or abstract objects relevant to the tasks. For instance, an
3
This level 2 is the only level distinguished by (Whitefield & Hill, 1994), on which figure
3.2 is based.
21

Chapter 3
object could be a text document and task behaviors could be typing, reading documents,
moving text etc.
A task is a set of behaviors directed towards a task goal. A task goal can be either a work
goal or an enabling goal. A work goal is related to a change in the work domain, an enabling
goal relates to a change in the structures designed to support the tasks, for instance loading
the text-processing software. A TM describes part of the TA domain. Unfortunately, the
different methods use different terminology, making comparison complicated.
Usually a TM will contain a description of the task behaviors. Methods such as Hierarchical
Task Analysis (HTA) (Annet & Duncan, 1967; Shepher, 1998), Groupware Task Analysis
(GTA) (van der Veer & van Welie, 1999), Méthode Analytique de Description (MAD)
Figure 3.3: Partial task decomposition using the GTA method. ‘And’ denotes
the subtask can be performed in parallel, ‘seq’ indicates the tasks must be
performed in sequence.
(Sebillotte, 1995), GOMS and User Interface Modeling (UIM) (Lif, 1999) uses a hierarchical
decomposition of the tasks, where top-level tasks are divided into subtasks. The lowest level
tasks are often called “unit tasks” 4 . These tasks can be further divided into actions. These
differ from tasks in that actions derive their meaning from the task they are a part of; they
have no meaning on their own. Users are seldom aware of the actions that are part of a
specific task. The hierarchy usually also includes information on the sequence in which the
tasks are executed, see for instance figure 3.3. Other methods, such as Contextual Design
(Beyer & Holtzblatt, 1998) do not use an explicit decomposition but focus more on
sequence, using an explicit sequence model.
In the previous description the term “task” was used, even though some of the methods
mentioned use the concept “goal”. This is no big difference, since tasks and goals in most
frameworks have a one-on-one relationship (van der Veer & van Welie, 1999).
Several methods, such as Contextual Design, GTA and UIM also include a description of the
objects of the work domain. Task can then be related to these objects. Furthermore, the three
methods mentioned here also allow tasks to be related to roles 5 that can be performed by
4 GTA distinguishes between "unit task", which is the lowest task level that people want to
consider in referring to their work, and "basic task", which is the unit level of task
delegation, is defined by the tool that is used in performing the work, like a single command
in a command driven application.
5 GTA and Contextual Design refer to sets of tasks that can be assigned to an actor as 'roles'.
UIM uses the term 'work situation' for a very similar concept.
22

Analysis
actors. Besides this, Contextual Design and GTA also support the modeling of the user’s
physical, conceptual, social and cultural environment.
Several methods for modeling collaborative work focus also on communication between
users. The Flow Model in Contextual Design models communication between users as part
of the flow from task to task.
3.2.2 Modeling user cognition
A slightly different approach than modeling the task is modeling the cognitive processes in
the head of the user when performing the task. Some of these TA methods deal specifically
with a specific section of level 3 of the TA domain as depicted in figure 3.2. Others also
include details of level 1 and 2 to relate the cognitive processes to higher order tasks.
The resulting insight into the skills and knowledge used to perform a task can be used to aid
in the design or evaluation of a user interface, as well as in determining the content of
training applications for conveying these skills and knowledge to the users. For instance,
Task Knowledge Structures (TKS) (Johnson & Johnson, 1991) describes how knowledge
about how to perform a task is stored in the memory of the user. Users maintain a
representation of the goals he or she is trying to achieve, similar to the task decomposition
described in the previous paragraph. Furthermore, the user has knowledge about objects and
the actions that can be performed on them as well as strategies and plans for meeting the
goals.
Cognitive Task Analysis (CTA) (Barnard & May, 1999) uses the theory of Interacting
Cognitive Subsystems (ICS). This theory represents the human information-processing
mechanism as a parallel architecture, built up from nine independent but architecturally
uniform subsystems. Each subsystems receives information (at an input array), stores it and
abstracts regularities over time (in an image record), and produces output (by a number of
parallel transformation processes). Although the subsystems are operationally identical, the
nature of the information that they receive and operate on differs, and this defines their
identity and role. In consequence, each subsystem can be thought of as dealing with a distinct
level of mental representation. Cognitive Task Models (CTM) describe the user’s cognitive
processing during task performance in terms of these ICS.
Intention
Sequence of actions
Execution of sequence
Goals
The The world world
Evaluation of interpretation
Interpreting the perception
Perceiving state of the world
Execution Evaluation
Figure 3.4: Norman’s seven stage cycle of interaction (Norman,
1988)
23

Chapter 3
CTA is resource intensive and has been of limited use to design practitioners. In response to
this, the Applied Cognitive Task Analysis (ACTA) method was proposed (Militello &
Hutton, 1998), which uses a practical approach to TA, linking sub-models of this method to
interview techniques. Using the ACTA method, an overview of the tasks is created as well as
the aspects of expertise needed to perform these tasks. By challenging the user with a task
scenario during an interview, insight can be gained into the cognitive processes of the user,
resulting in a list of the cognitive demands of the task.
A generic model of user cognition during interaction with computer systems or any other
part of a person’s environment is proposed by (Norman, 1988), as depicted in Figure 3.4.
The user responds to changes in the world. In the case of collaborative work, such as in
VRET, these changes can be caused by other users or by the computer. The different stages
involved in the user interaction are: establishing goals, forming the intention, specifying the
action sequence, executing the sequence, perceiving the system state, interpreting the system
state and evaluating the system state.
3.2.3 Modeling the user interface
In addition to modeling the task and possibly the cognitive processes of the user it is possible
to create a model of the proposed or existing user interface to understand the implications of
the specific user interface design for the usability of the system.
A model of the proposed interface can contain only the functionality of the system, where
implementation details and details of the underlying hardware are suppressed. Terms used to
denote such a model are the User’s Virtual Machine (van der Veer & van Welie, 1999) or,
according to the Contextual Design method, the User Environment Design (UED) (Beyer &
Holtzblatt, 1998). The UED uses the concept of focus areas; coherent ‘places’ in the system
where only those functions needed for performing a specific task are present.
The method of Foley (Foley et al., 1996) describes the functional design of a user interface at
four levels: the conceptual design, semantic design, syntactical design and lexical design.
The conceptual design describes the mental model that the user must be able to form. It
describes any metaphors used and how these metaphors are coupled to parts of the user
interface. The semantic design describes what will be the result of specific user actions in the
interface. The syntactical design describes the user interface in terms of state transitions, and
the lexical design describes the input and output of the system, in terms of input devices and
layout and appearance of the user interface.
Instead of modeling the functional requirements, EID (Ecological Interface Design) focuses
on the information that should be presented in the user interface to aid the users in their
decision making. Aimed at systems that are used to control complicated processes such as
nuclear reactors, EID models the process to be controlled at different levels of abstraction.
By selecting the right level that is necessary for the type of problem solving applied by the
user, those representations that have to be incorporated into the UI are found (Burnst, 2000;
Vicente et al., 1995).
The Language / Action Perspective (Winograd, 1987) focuses specifically on the
communication between users and between user and computer, the latter being determined
for a great deal by the design of the UI. According to the Language / Action Perspective,
using language is similar to performing an action; asking someone to do something is similar
to pressing a button: it changes the state of the world. Instead of looking at the form of
language, the Language / Action Perspective emphasizes the pragmatics of language, what
people do with language. People perform Speech Acts with language (Searle, 1969; Searle,
24

Analysis
1979; Habermas, 1981; Dietz & Widdershoven, 1991), which can be analyzed to determine
the structure of a user's task.
Layered Protocol Theory (LPT) (Farrel et al., 1999; Taylor et al., 1999) considers HCI to be
equivalent to a dialog between humans. In a dialog, people act to reduce the difference
between a perceived current state and desired condition for that state, changing how they act
if the perceived state is not converging to the desired state. This will take place at various
levels of abstraction. For instance, at a high level the user may desire a file to be transported
to a different location on the computer. On a lower level the user will desire the computer to
know which file is the subject of the operation.
Communication takes place over certain media, such as verbal communication through
sound or text input through a keyboard. The Comms Usage Diagram (CUD) as proposed by
(Watts & Monk, 1998) describes the behavior of users, the communication they have and the
media they use.
A user interface of a VE usually incorporates both discrete and continuous in- and output.
(Smith & Duke, 1999) propose a notation based on Petri-Nets, including modeling
components representing continuous processes.
3.2.4 Applying current Task Analysis methods to VRET
In the previous paragraph we have reviewed a wide variety of modeling approaches used in
HCI design. The question now remains: which approach, if any, are suited for applications
such as VRET? For this, we will have to look at the characteristics of these applications
Collaborative work
In VRET, there are two users who are working together: the therapist and the patient. We
therefore need an approach that takes multiple users into consideration such as Contextual
Design, GTA or UIM. These methods usually consider a work situation where users
cooperate within a relatively large time frame, sometimes a-synchronous. Communication
between users is depicted using a flow diagram or similar. However, in VRET we see a very
close collaboration between users in a short time frame, where real-time communications
play an important role and users work in synchrony. Furthermore, this communication does
not necessarily follow a predetermined sequence, as will become evident later on. Applying a
flow diagram might therefore not be an optimal choice in this situation. A very promising
approach is the Language / Action Perspective, which focuses specifically on the
communication between user and computer, but could also be used to include human-tohuman
communication.
Implementation dependency
VR is a type of highly interactive UI that is still under development. There are, at this
moment, no standard ways of interacting with immersive VR systems. Many of the
intricacies of the HCI in VRET therefore lie in determining the correct interaction techniques
and specifics of the implementation of those techniques. In terms of the method of Foley
(Foley et al., 1996), we need to look not only at the conceptual and semantic design, but also
at the syntactic and lexical design.
As stated before, there are those who believe that a TM should be implementation
independent and only include the user's goals. However, for applications such as VRET, this
will not do. We need to model how specific design implementations influence the user's
work. We will need to look at detail level 3 of the TA domain, modeling the user’s
25

Chapter 3
interactions and whether these meet the requirements of the cognitive processes of the user.
At this detail level we cannot use the same model for the existing and the future situation,
since these will have different implementation specific details. We will need two models:
TM1 and TM2.
Multi-modality
Interaction in these applications uses multiple modalities. Users communicate with each
other through use of the computer, and in this case also directly by talking. The computer
responds to different types of inputs, such as the keyboard, joystick, but also headtracking,
and produces output in different types of modalities such as vision, sound or even haptics.
These modalities cannot be viewed separately; they are co-dependent and influence one
another. We therefore need a method that can model these modalities. Again, the Language /
Action Perspective looks promising since it considers communication in different modalities
to be equal. However, different modalities might afford different possibilities for
communication and we should therefore look at which modality is used for which type of
communication, possibly similar to the CUD.
Level 1
Level 2
Level 3
Modelling
User
Cognition
Modelling
the UI
Modelling
Tasks
Pragmatics
Semantics
Knowledge
User(s)
Computer(s)
Cognitions
Syntax
Lexicon
User Interface
Tasks Domain
Objects
Figure 3.5 shows what part of the potential domain of TA is covered by the various
discussed methods. Most TA methods limit themselves to detail level 1 and 2. The reason for
this is that in reality users perform a great number of tasks that are interrelated, but these
tasks themselves are often straightforward and uncomplicated. An HCI-designer is therefore
Goals
Media
Objects
Figure 3.5: Domains of the various TA methods as subsets of
the complete potential domain of TA.
best served with an overview of all the tasks that need to be supported.
In VRET there are a limited number of tasks being performed by the users, however, these
tasks are of a complex nature, taking place in a close collaborative setting and requiring
communication over various modalities. This requires a detail analysis of how tasks are
performed, requiring a TA of detail level 3. There are several TA methods that cover part of
this detail level, but none that can show a complete picture of how the system and the users
26

Analysis
influence each other. To best facilitate the design of user interfaces for VRET, we will most
likely need a combination of the existing methods at level 3.
3.3 Adapting existing TA methods
Similar to most TA methods, we can include a task hierarchy into our analysis. This will
allow us to include low-level details of the implementation and place these into the context
of structures at a higher level of abstraction. Because we are looking explicitly at the
implementation level of our work situation, it is a good idea to create a distinction between
those parts of the task hierarchy that are dependent on the current task implementation and
those that are not and should therefore hold for other task implementations as well.
The definition of the various tasks and actions could be performed arbitrarily, at the analists
discretion. However, to ensure that the analysis becomes less dependend on the individual
analist, an existing framework can be used. One such framework that can be applied in this
case is the Language / Action Perspective By viewing the interaction as part of a
conversation between users and the computer, we can incorporate the collaborative aspects
of these types of applications. Using speech act theory, we can categorize the actions of users
and computers in several categories. (Habermas, 1981; Dietz & Widdershoven, 1991):
• Imperative: ordering the listener to perform an action, usually implicitely stating that the
speaker holds the power over the listener to demand this action.
• Constative: commit the speaker to something being the case - to the truth of the
expressed proposition
• Regulative: either requesting the listener to perform a certain action, or promising that
the speaker will perform an action.
• Expressive: express a psychological state about a state of affairs (e.g., apologizing and
praising)
These categories can help in identifying the lowest level of our task hierarchy and can aid in
defining these tasks in an objective fashion, in other words, making the particular choice of
definitions independent of a particular analyst.
Speech acts are not unrelated events, but participate in larger conversational structures
(Floris & Ludlow, 1981). An analysis of the sequence in which events occur can provide
insight into the typical structure of a therapy session. Several methods exist for uncovering
the sequence of events in human-computer interaction. The most commonly used are
cognitive analysis and graph-matrix analysis (McGrew, 1997). As will become clear later on,
the interaction in VRET systems is of such an unpredictable nature that cognitive analysis,
where the observer must aggregate, analyze and abstract the task structure, is not very
suitable. The more quantitative approach of graph-matrix analysis can be used in this case.
The advantage of a task hierarchy is that we can use it to check whether a design meets all
functional requirements. We can do this by determining whether all high level
implementation-independent goals are supported by lower level tasks and whether these
tasks can be performed with the actions available to the user. However, we are also interested
in how implementation specific details affect the HCI and for this we will first need a closer
look at the way users interact with systems, and especially at the user’s cognitions.
27

Chapter 3
3.3.1 Interaction cycle
For this we can use the interaction cycle proposed by Norman and described in paragraph
3.2.2. as a cycle where the user evaluates the state of the world, compares this to certain
goals that the user has and executes certain actions to alter the state of the world. In this
context, Norman states that there are two gulfs that need to be bridged by the user interface
of a system to make it usable. The gulf of execution refers to the user not knowing how to
perform an action, the gulf of evaluation refers to the user not being able to evaluate the state
of the system.
For the gulf of execution to be bridged, the system should first of all provide the required
functionality, i.e. support a specific user action. This can be checked using the task hierarchy
as mentioned before. Furthermore, the user must also know that the action is available and
know how to execute the action. This knowledge has to be derived from a source of
information and we will call this type of information procedural information.
To bridge the gulf of evaluation, the system should provide information about the state of the
system and results of user actions. We shall call this type of information state information.
Furthermore, the user must be able to interpret this information and evaluate it in terms of the
user’s goals and intentions.
3.3.2 Procedural information
Procedural information refers to the information
needed to understand how to operate the system.
It is part of what (Norman, 1988) calls ‘the
user’s model’. The user’s model is usually
implicit and is based on the system image. The
system image consists not only of the
appearance of the computer and the content of
the screen, but also of any manuals or training
that are provided with the system.
The system and the system image are based on
the design of the system. We must therefore
ensure that our design creates a system image
from which the user can infer a correct user’s
model. One way of doing this would be to create
very accurate manuals and provide extensive
design
model
Designer
System
system
image
User
user’s
model
Figure 3.6: The designer creates the
system after a design model. The user
infers a model of the system based on the
system image (Norman, 1988)
training. However, part of our definition of usability is that the system is easy to learn and
therefore the user should be able to infer the correct model with minimal use of either.
Humans and other animals have the ability to infer what actions are possible in their
environment. In everyday life, we have learned to recognize the affordances (Gibson, 1979),
or possibilities and opportunities of our environment. For instance, the ground affords
walking, a chasm affords falling and hurting, an apple might afford eating and a tiger affords
being eaten. We can use this knowledge to hint users at the affordances of the system
(Norman, 1993). For instance, an empty text box with a blinking cursor affords entering text
or a button affords being pressed. We should therefore extend our task model to check
whether the system provides perceivable affordances to the user for each action.
28

Analysis
3.3.3 State information
Not only does the user need to know how the system works in general, the user also needs to
know what the state of the system is at a specific moment in time. For instance, the therapist
would need to know not only how to maneuver a patient through a VE, but also what part of
the VE the patient is seeing right now. Also, the therapist would need to know what the fear
level of the patient is at this moment. For this, the user will need information in a timely
fashion and this information will have to be communicated to the user through use of a
medium such as the computer screen or the verbal comments of the other user. In our model,
we can keep track of which medium provides which state information.
State information can be stored in the head, but it can also be stored in the world. This is
done by using cognitive artifacts (Norman, 1993): elements of the environment to which a
meaning is assigned by the user. These elements can be arbitrary objects in the environment
but they can also be part of the user interface for the system, such as a throttle lever in an
airplane which serves to operate the throttle and at the same time reminds the pilot of the
selected throttle level.
These cognitive artifacts can, at the same time, serve as communication tools with others.
For instance, the throttle lever mentioned in the previous example can also inform the copilot
of the throttle level selected by the pilot. In such a situation, the cognitive activity can
be said to be distributed over several persons and objects, which is called Distributed
Cognition (DC) (Hutchins, 1995).
The concept of DC can be used in HCI design to improve usability by allowing designers to
identify those user interface components that might assist cognitive processing as cognitive
artifacts (Halverson, 1994) or even serve as means of communication between users.
3.3.4 Overview
The proposed TA method starts with an analysis of the current way of working by classifying
events in the HCI using the Language / Action perspective. The relationships between these
events can be made clear by analyzing the sequence in which these events typically occur.
By relating these task events to higher-level goals that can be determined with the help of
therapists, a task hierarchy can be constructed. For each task the information needs can then
be defined, both for procedural and state information. By comparing these information needs
to the information supplied by the system, we can determine whether these needs are met. If
not, there is a problem in either bridging the gulf of execution or gulf of evaluation, and we
have identified a possible source of usability problems. The state information flows can also
be included in the task hierarchy by creating a diagram with a representations of both the
task hierarchy and the several media through which communication takes place. Such
diagrams are shown in figures 3.19 through 3.24.
The TA method described here includes a hierarchical task decomposition similar to HTA
and GTA, covering detail level 1 and 2 of the TA domain. Furthermore, several areas of level
3 are also covered as depicted in figure 3.7. The interaction cycle proposed by Norman is
used to understand the user’s cognitions, the Language / Action Perspective is used to
represent both the human-computer interaction and the interaction between both users. The
media used for these interactions or communications is also modeled, using a simple notation
technique.
In the proposed TA method, these level 3 components are furthermore related to one another
and to the level 1 and 2 task hierarchy. The user’s cognitions are related to the goals or tasks
that a user is trying to achieve, and we can try to determine whether the interaction as
29

Chapter 3
described using the Language /
Action perspective provides enough
procedural information to bridge the
gulf of execution and sufficient state
information to bridge the gulf of
evaluation, thus enabling the user to
have the proper cognitions to perform
his or her tasks. These interactions
can then be related to the different
media available to the users.
The proposed TA method should give
insight into the intricacies of the
VRET process and possibly provide
Interaction
cycle
Language/action
perspective
Pragmatics
Semantics
Knowledge
Cognitions
Syntax
Lexicon
User Interface
hints and clues to any usability problems that exist within a current HCI design. To test
whether it is a useful method, we will apply it to the case of acrophobia treatment using VR,
which will be described in paragraph 3.5 to 3.8 of this chapter. But first of all we will
consider how to gather the data for our analysis.
Goals
Media
Media
usage
Figure 3.7: Areas covered by the proposed TA
method components.
3.4 Gathering data
Because the designer is usually not an expert in the domain of the user, task analysis cannot
be based on assumptions of the designer. To create a TA of the VRET process, data should
be gathered about the way the user really works by means that ensure the resulting
description of the work is valid.
3.4.1 HCI Data gathering techniques
Several general techniques have been reported in the literature, the most popular being:
Interviews
Open interviews usually start with a single open question such as: “tell me about your work”,
followed by a more or less informal discussion. In structured interviews the interviewer uses
a predetermined list of questions. An alternative form is the semidirected interview
(Sebillotte, 1995). Here, the interview can start with a predetermined question, and based on
the answer, can formulate a new question, following predetermined rules.
The advantage of interviews is that they are a relatively easy and cheap means of getting a
wide variety of data. Furthermore, interviews might be the only viable option when
observation of the user at work is not possible, as is the case in traditional phobia treatment
because of patient confidentiality.
A disadvantage of interviews is the fact that users may not be able to correctly verbalize their
knowledge; users are not always consciously aware of all low-level details in their work.
Observational techniques
Observations can be made of the user at work. People react differently when they are being
observed so the observations should be as unobtrusive as possible. Instead of direct
observation one can also use video-recordings or time-lapse photography. To gain an
understanding of the cognitive processes involved in the task, the “think-aloud” protocol can
be used. Here, users are asked to verbalize their thoughts while using the system.
30

Analysis
Unfortunately, this will most likely affect the users in their work as they are constantly
reminded of the observer. To reduce this effect, the “cooperative evaluation” protocol has
been proposed. Here, two users are required to use the system simultaneously, forcing them
to communicate about the system, without the users being aware of an observer. For
applications where the users already need to communicate with each other extensively such
as VRET, both the “think-aloud” protocol and the “cooperative evaluation” protocol are
unsuitable however, because these protocols most likely will interfere with the normal
communication.
(Johnson, 1998) furthermore states that another disadvantage for using the “think-aloud”
protocol when applied to VR-systems, is that users will be unable to explain what contributes
to a successful VE interface because they have difficulty in trying to articulate a “3D
language”. However, other researchers found no such problems (Marsh, 1999; Kaur, 1998).
Another way to gain more in-depth insight into the users and their environment is through
ethnographic methods. Here, the designer temporarily becomes part of the user group,
usually as an apprentice. The goal is to study users in their natural behavior, to uncover all
relevant phenomena in the task domain that are not explicitly verbalizable.
Contextual inquiry
Contextual inquiry (Beyer & Holtzblatt, 1998) can be called a combination of interview and
observation. Here, the user is interviewed while at work, where he or she can demonstrate
how tasks are performed. Instead of using a prescribed path for the interview, it is advised to
shape the interview by using a relationship model. One such model is the master/apprentice
model, where the user must take on the role of master and the designer must take on the role
of apprentice.
Trace-analysis
For some activities, it is not possible to make observations, for instance because the activity
involves the confidential relationship between therapist and client or because the activity
only occurs in case of emergencies. In such a case, indirect observation might still be made
based on certain documents such as logs.
In practice, data gathering will be performed using several of the aforementioned techniques.
We will now need to consider which techniques are appropriate for VRET.
3.4.2 Data gathering in VRET
As stated at the beginning of this chapter, only a few early adopters are already using VRET
in practice today. It is therefore a problem to try and analyze real VRET practice. To create a
TA of VRET, we will therefore use two different sources of information:
1. Current VRET practice in the laboratory.
2. Current traditional exposure therapy.
For each source of information, different data gathering techniques will be used
Ad.1: Current VRET practice in the laboratory
An important source of information will be the VRET as performed in our laboratory and the
laboratory of our partner, the University of Amsterdam. Here, therapists are treating patients
31

Chapter 3
as part of ongoing investigations into the effectiveness of VRET. The advantage of having
such a laboratory setting is that we can have full access to therapists, patients and the
computer system. However, there are still limitations as to the possibilities for data
gathering. As discussed earlier, the “think-aloud” protocol and the “cooperative evaluation”
protocol are unsuitable for collaborative systems such as VRET, since these protocols are
very likely to interfere with the normal communication between users. We will therefore
contend with simple video-observations. These should be unobtrusive, and not effecting the
therapy process. Furthermore, we question the therapists about their actions during VRET.
The system used should be representative for the state-of-the-art currently in use in
laboratories (and possibly practices) around the world. This way, we should be able to
generalize the findings in our laboratory settings to other systems as well. For this, we have
used a web-based questionnaire to create a survey of the characteristics of systems currently
available on the market. The system we used was designed to be flexible in terms of its user
interface, and it was configured to be similar to the state-of-the-art found in the survey.
Ad.2: Current traditional exposure therapy
There exists a great deal of literature about the theory and practice of traditional therapy for
phobias. By reviewing guidelines set forth in this literature regarding the therapy process
itself, we can gain insight into the way traditional therapy is performed.
In reality, therapy practice might not exactly follow literature. We also need insight into the
current practice of in vivo exposure therapy to create a system that therapists will actually
use in practice. This source of information is especially important since it can provide insight
into the context in which a future system will be used.
Unfortunately, due to patient confidentiality it is not possible to observe actual therapy
sessions. We have therefore contended with interviewing several therapists from different
institutions. For this we have used semi-directed interviews.
3.5 Current VRET practice
3.5.1 Current state-of-the-art
Even though VRET is still in an experimental phase, already several groups have produced
and are selling VR-systems for treatment of phobias. Through a web-based questionnaire,
personal contact with the developers and hands-on experience with several of the systems, an
inventory of the user interfaces has been created. In table 3.1 an overview of these systems is
presented with a reference to the website about these systems and a list of the phobias that
the systems were designed for.
Table 3.2 shows several specifics of the user interface for the patient. All systems use a
Head-Mounted Display (HMD) and offer at least visual and auditory stimuli. Some systems
can generate vibration to simulate for instance engine vibrations in a car or airplane. Most
HMDs are tracked with 3 Degrees of Freedom (DoF), measuring rotation around 3 axis. Two
of the systems incorporate 6 DoF tracking, also measuring translation in 3 dimensions.
32

Analysis
Name system URL website Phobias (other disorders)
Hanyang University bme.hanyang.ac.kr/vr Acrophobia, Fear of driving, Fear of
speaking, Agoraphobia
Virtually Better www.virtuallybetter.com Acrophobia, Fear of flying, Fear of
speaking, Agoraphobia, Fear of
thunderstorms, (Post traumatic stress
disorder for Vietnam veterans)
VRHealth www.vrhealth.com Claustrophobia, (Panic disorders),
(Eating disorders)
Previ www.previsl.com Fear of Flying, Claustrophobia, (Eating
disorders)
DriVR www.driVR.com Fear of Driving
VRT-2002 science.kennesaw.edu/~
mnorth/vrt1/vrt1.html
Acrophobia, Fear of flying, Agoraphobia,
(Obsessive Compulsive
disorders), (Attention Deficit
Disorders)
CYBERmed www.insight.co.at Fear of flying
Table 3.1: Overview of the commercial VR-systems currently available for phobia
treatment, with reference to the website about these systems and a list of the phobias (and
other mental disorders) that these systems are designed for.
Name system Display Modalities Headtracking Other input devices
Hanyang
University
HMD Sight
Sound
Vibration
Virtually Better HMD Sight
Sound
Vibration
VRHealth HMD Sight
Sound
Previ HMD Sight
Sound
DriVR HMD Sight
Sound
3 DoF Tracked device
Driving controls
6 DoF Tracked device
Driving controls
3 DoF Joypad
3 DoF Mouse
3 DoF Driving Controls
VRT-2002 HMD Sight
Sound
Vibration
6 DoF Tracked device
CYBERmed HMD Sight
Sound
3 DoF
Table 3.2: The user interface for the patient, with the type of display, modalities addressed
by the system, type of headtracking (DoF = Degrees of Freedom) and other input devices.
Table 3.3 shows some characteristics of the user interface for the therapist. Because all
systems use an HMD, the therapist cannot look at the same display as the patient. Instead a
monitor is provided which shows the same image displayed in the HMD. Both therapist and
patient will therefore see the VE from the patient’s viewpoint. Usually the main input is the
33

Chapter 3
keyboard. Before the therapy session, the therapist can alter certain parameters of the VE.
During the exposure session, the therapist can alter certain elements in the VE. We see
several systems where the therapist can control the viewpoint of the patient and several
systems where the patient is in control of his or her viewpoint.
34
Name system Display Viewpoint Input devices Controls
Hanyang
University
Virtually
Better
Different
Display than
patient
Different
Display than
patient
VRHealth Different
Display than
patient
Previ Different
Display than
patient
DriVR Different
Display than
patient
VRT-2002 Different
Display than
patient
CYBERmed Different
Display than
patient
Same
viewpoint as
patient
Same
viewpoint as
patient
Same
viewpoint as
patient
Same
viewpoint as
patient
Same
viewpoint as
patient
Same
viewpoint as
patient
Same
viewpoint as
patient
Keyboard
Joystick
Tracked
device
Elements
Pre-VE
modifications
Keyboard Elements
Viewpoint
Pre-VE
modifications
Keyboard None
Keyboard Elements
Viewpoint
Pre-VE
Keyboard
Joystick
Tracked
device
WIMP (pre-
VE)
Keyboard
Joystick
Tracked
device
WIMP (pre-
VE)
? ?
modifications
Pre-VE
modifications
Elements
Viewpoint
Pre-VE
modifications
Table 3.3: User interface for the therapist, with type of display, viewpoint displayed, types
of input devices and type of control the therapist has.
3.5.2 Virtual Reality testbed
To investigate the human-computer interaction in current VRET-systems and to evaluate
modifications to the user interfaces a testbed was created (Schuemie & van der Mast, 2001).
The system was designed to meet the following requirements:
1. The system had to be easy to modify. It should be possible to test various types of
interfaces for both therapist and patient.
2. The system must have the potential for effective therapy. Unfortunately, very little is yet
known about the specific requirements needed for evoking phobic responses in VR. We

Analysis
have assumed that stereoscopy would be important for acrophobia, since this greatly
enhances the perception of depth.
3. The system should not be too expensive. Since the system is intended for use by
therapists, the system should be affordable for institutions with a small budget
The choice was made for a PCbased
system with off-the-shelve
components. The display was a
Visette Pro stereoscopic HMD
with a field of view of 70 degrees
diagonal. The position of the
HMD was tracked using the
Ascension Flock of Birds, a six
DoF tracker. The two images,
respectively for the left and right
eye, were generated using two
Intergraph Intense3D Pro 2200s
cards with 16 megabytes of
texture memory each. These cards
were chosen because the output of
these cards can be synchronized
using a so-called GenLock, a
necessary requirement for most
TCP/IP
Delphi
WorldUp
OpenGL
Windows NT
Delphi
WorldUp
Direct 3D
Windows 98
Patient’s computer
Therapist’s computer
Intense 3D Pro
GenLock
Intense 3D Pro
keyboard, mouse
joystick
3D card
Flock
of Birds
Figure 3.8: Overview of the testbed configuration used
for the VRET.
HMDs. During the years of the research described in this dissertation, these cards were
replaced with a single, much faster graphics card: the 3D-Labs Oxygen GVX420.
A second computer was used to display the user interface for the therapist, and the two
computers communicated over a standard Ethernet network using the TCP/IP protocol. This
made it possible to connect the two computers over great distances, but this was never used
in our experiments.
The VR software package used was WorldUp R4 by Sense8. This high level toolkit made it
possible to develop new VEs within several days. Furthermore, it has support for a wide
variety of input and output devices. For extended functionality, such as two-dimensional
graphical user interfaces, network communication, database access and complicated
computations, Borland Delphi 5 was used to create Dynamic Link Libraries (DLLs) which
were called from WorldUp. An overview of the system is presented in figure 3.8.
3.5.3 Case: acrophobia treatment
To investigate the human-computer interaction in VRET, therapists of the University of
Amsterdam used the system to treat patients clinically diagnosed as having fear of heights.
These treatments were also part of two investigations in comparing VRET to exposure in
vivo (Emmelkamp et al., 2002; Krijn et al., in preparation). The choice for acrophobia was
based on the fact that this is one of the less complicated phobias.
Subjects
The therapists involved were four master students and a PhD student in clinical psychology,
who already had some experience in phobia treatment. Patients were individuals suffering
from acrophobia as their main complaint, who referred themselves for treatment after
35

Chapter 3
announcements in the national media offering
psychological treatment for acrophobia. To be included
in the study, patients had to fulfill the DSM-IV
(American Psychiatric Association, 1994) criteria of
acrophobia, acrophobia had to be the main problem,
and patients had to be unable to complete a Behavioral
Avoidance Test. In all, 68 subjects completed the VR
treatment.
VR system
The user interfaces for patient and therapist were
similar to the current state-of-the-art. The patient wore
the HMD with six DoF tracking. Besides visual
stimulation, auditory stimuli were provided in the form
of wind and people talking. The patients could only
move their viewpoints as far as the limited range of the
tracker and the limited range of physical movement in
reality allowed.
Further movement of the patient’s viewpoint could be
performed by the therapist by use of a joystick or by
using an autopilot; Pressing a number on the keyboard
resulted in the system automatically moving the patient
from the current location to the location corresponding
to the number pressed. The video signal for the HMD
was also displayed on a monitor, allowing the therapist
to see what the patient could see.
The three virtual environments were based on real
situations. The first environment was Magnaplaza, a
mall in the center of Amsterdam, which features
several floors and escalators, as shown in figure 3.10.
The second environment included a firestair besides a
three-story building, as shown in figure 3.11. The last
environment was the rooftop terrace of a seven-story
building of the University of Amsterdam, overlooking
the Magnaplaza mall across the street, as depicted in
figure 3.12. None of these environments contained
elements which could be changed by the therapist.
3.5.4 Interpreting observations
20 treatment sessions of the case described in the
previous paragraphs were recorded on video, allowing
the observations to be made in an unobtrusive way.
Subjects were asked for permission to make the
recordings before the therapy.
The observations were classified using the Language /
Action Perspective. As such, pressing a button can be
similar to uttering a sentence, allowing us to include
36
Figure 3.9: The physical setup as
used for the therapy.
Figure 3.10: VE of Magnaplaza
mall.
Figure 3.11:VE of firestairs.
Figure 3.12: VE of rooftop terrace.

Analysis
several modalities of communication in our analysis.
Based on a preliminary video analysis of several therapy sessions, the following speech acts
were identified by the author of this dissertation:
For the therapist:
1. Regulative: Asking the patient to look in a certain direction
2. Regulative: Asking the patient to move in a direction
3. Regulative: Asking the patient to change their posture (e.g., release their hands from the
railing)
4. Regulative: Asking the patient to report their level of fear. Often the patient was
instructed to use a scale from one to ten to report their fear, a so-called Subjective Unit
of Discomfort (SUD) (Wolpe, 1958)
5. Imperative: Instructing the computer to change the location of the patient in the VE,
either through use of the joystick or keyboard
6. Constative: Inform the patient about the rationale of the therapy
7. Constative: Inform the patient about the VE
8. Constative: Inform the patient that the therapist will change the location of patient in the
VE
For the patient:
9. Regulative: Ask a question to the therapist
10. Imperative: Control the viewpoint in the VE through use of the tracking device
11. Expressive: Indicating the level of fear the patient is currently experiencing. This can be
either verbally or through a change in posture (e.g., clutching the railing, crouching)
For the computer:
12. Constative: Display of the current state of the VE from the patient’s viewpoint to the
patient through the HMD
13. Constative: Display of the current state of the VE from the patient’s viewpoint to the
therapist on a monitor
Figure 3.13 shows the lines of communication
between the users and the computer. The dotted
lines indicate that the communication is of a
continuous nature, making it difficult to discern
discrete events. For now, we have therefore not
taken this communication into account in our
analysis of the sequence of events. Although
controlling the location of the patient in the VE
using the joystick (event type 5) can be
continuous in nature, the therapists used this
only in a discrete fashion, moving the patient to
a certain location and then stopping the
movement. This allowed us to take these types
of events into consideration. Sometimes during
1,2,3,4,6,7,8
Patient Therapist
9,11
10
5
12 13
Computer
Figure 3.13: Communication between
users and the computer. Dotted lines
indicate a continuous flow of information.
Numbers indicate the category of events.
the therapy a long period elapsed while no discrete event occurred. When such a period
lasted longer than 15 seconds, this was recorded as a ‘silence’.
37

Chapter 3
3.5.5 Identifying sequence
To reduce the impact of simulator sickness, therapy sessions were divided into two segments
of approximately 20 minutes. The video recordings of an arbitrary number (13) of such
segments, including all five different therapists and seven different patients, were observed
for our TA by the author of this dissertation. In all, 1420 events were classified using the
categories mentioned above.
1.Regulative: Look 242
2.Regulative: Move 50 50
3.Regulative: Posture 35 35
4.Regulative: Report 306
5.Imperative: Control VE 161
6.Constative: Rationale 14 14
7.Constative: VE 18 18
8.Constative: Move
Patient
109
9.Regulative: Question 17 17
11:Expressive: Fear 374
Silence 92
Figure 3.14: Histogram of the speech acts that occurred during therapy.
92
Only a small fraction of the recorded events occurred simultaneously. A small number of
events overlapped, for instance the therapist sometimes asked a patient to look over the
railing while still moving towards that railing. In such cases, the start of the event was used
to determine the sequence.
47
Silence
92
80
57
109
T. regulative:
report fear
P.expressive:
fear
28
300
94
306
161
62
242
T.constative:
movement
374 109
101
T. imperative:
Control VE
83
T. regulative:
look
Figure 3.15: Graph showing the same transitions between events as shown in table 3.4.
However, only the 10 most prevalent transition types are shown to reduce visual complexity.
The numbers near the arrows indicate the absolute number of transitions, the numbers in the
boxes indicate the absolute number of times the event occurred.
38
161
242
306
117
374

Therapist
1. Regulative: Look
2. Regulative: Move
3. Regulative: Posture
Analysis
Figure 3.14 shows a histogram of the occurrence of events during the therapy sessions. Here,
we can already see that there are several types of events that are predominant during therapy.
To gain insight into the sequence of events we can look at the transitions from one event to
the other. Table 3.4 shows the absolute number of transitions between the events during the
therapy sessions. Most events are related to a large portion of the other types of events.
4. Regulative: Report
5. Regulative: Control VE
6. Constative: Rationale
7. Constative: VE
8. Constative: Move
Patient
9. Regulative: Question
Therapist
1. Regulative: Look 11 14 5 117 24 1 0 15 5 22 28
2. Regulative: Move 15 0 3 18 4 0 0 4 0 5 1
3. Regulative: Posture 9 3 0 14 4 0 0 2 0 2 1
4. Regulative: Report 1 0 0 2 0 0 0 0 3 300 0
5. Imperative: Control VE 83 18 6 18 10 2 2 10 0 10 2
6. Constative: Rationale 2 0 0 5 0 0 0 6 0 1 0
7. Constative: VE 5 0 1 4 2 2 0 1 0 0 3
8. Constative: Move 5 0 1 1 101 0 0 1 0 0 0
Patient
9. Regulative: Question 1 0 0 0 0 3 13 0 0 0 0
11.Expressive: Fear 94 13 18 80 12 6 3 62 7 22 57
Silence 15 2 1 47 4 0 0 8 2 13 0
Table 3.4: Absolute number of transitions between events. Left are the preceding events, on
the top are the following events.
In all 76 different types of transitions have been observed. The number of types of transitions
displayed should be smaller to reduce the visual complexity of the resulting transition graph.
Analysis showed that the ten most prevalent types of transitions accounted for 68% of all
transitions. Figure 3.15 shows the transition graph with only these 10 transitions. Now, we
can get a good idea of the structure of a typical therapy session:
Often during the therapy, the therapist will ask the patient to report her/his fear level. Based
on the response of the patient, the therapist will either choose to
1. ask the patient to further elaborate on what he or she is experiencing. Further analysis
showed that when a patient‘s report of his/her fear level is immediately followed by
another question about the patient’s fear, it is slightly more likely for this second
question to be of a more qualitative nature (47 transitions), instead of simply asking the
patient how afraid they are at that moment (33 transitions)
2. let the patient remain in the current situation (to allow the fear to diminish further),
which can often be identified by a silence event
3. move the patient or ask the patient to look in another direction, usually with the goal to
increase the fear the patient is experiencing
11.Expressive: Fear
Silence
39

Chapter 3
When the therapist chooses to move the patient to another location, this is often preceded by
an announcement by the therapist that he or she will do so. Further analysis showed the
joystick to be preferred over the autopilot: only 19% of the movement acts were performed
with the autopilot. After having moved the patient to a new position, the therapist almost
always asks the patient to look in a certain direction, followed by a request for the patient to
report his or her level of fear.
Apart from looking at the absolute number of transitions, we can also look at the relative
number of transitions. Table 3.5 shows the fractions of transitions from one event to the
other, compared to the total number of transitions from that event. These fractions can be
seen as probabilities, showing that very few transition types can clearly be predicted.
However, the transitions from some of the most prevalent event types are far more
deterministic, such as the transition from event type 4 to event type 11, which is the subconversational
structure of the therapist trying to determine the fear level of the patient.
Therapist
1. Regulative: Look
2. Regulative: Move
3. Regulative: Posture
Therapist
1. Regulative: Look 5 6 2 48 10 0 0 6 2 9 12
2. Regulative: Move 30 0 6 36 8 0 0 8 0 1 2
3. Regulative: Posture 26 9 0 40 11 0 0 6 0 6 3
4. Regulative: Report 0 0 0 1 0 0 0 0 1 98 0
5. Imperative: Control VE 52 11 4 11 6 1 1 6 0 6 1
6. Constative: Rationale 14 0 0 36 0 0 0 43 0 7 0
7. Constative: VE 28 0 6 22 11 11 0 6 0 0 17
8. Constative: Move 5 0 1 1 93 0 0 1 0 0 0
Patient
9. Regulative: Question 6 0 0 0 0 18 76 0 0 0 0
11.Expressive: Fear 25 3 5 21 3 2 1 17 2 6 15
Silence 16 2 1 51 4 0 0 9 2 14 0
Table 3.5: Transition probabilities in percentages between events: probability of the top
event occurring next, given the fact that the left event has just occurred. Grey indicates the
probability is higher than 50%
3.5.6 Conclusions
In the previous paragraph the typical course of a therapy session has been analyzed and
described. In this sense the description is similar to a scenario as often used in HCI design.
However, our analysis also includes detailed information of the relative occurrence of the
various sequences of events, and therefore provides a more thorough basis for the design
process. It will form an important part of our overall task model as described in paragraph
40
4. Regulative: Report
5. Regulative: Control VE
6. Constative: Rationale
7. Constative: VE
8. Constative: Move
Patient
9. Regulative: Question
11.Expressive: Fear
Silence

Analysis
3.7, and since it provides quantitative information concerning the various task elements, it
can be used to identify opportunities for increasing the usability such as frequent repetitive
tasks that could be automated or determining which improvement will have the highest
impact on overall system usability.
3.6 Current In Vivo therapy
Most phobias can be treated effectively with exposure therapy. (Bouman et al., 1992) Here,
the patient is exposed to stimuli that produces the anxiety and by a process known as
habituation the fear will gradually diminish. Usually, the patient is gradually exposed to
more fearful stimuli. Until recently, the stimuli could be either imaginary or real (in vivo).
A review of literature shows gradual exposure therapy in general to consist of the following
steps: (Bouman et al., 1992) (Davidson & Neale, 1996) (Orlemans et al., 1995)
1. Function analysis. Determining the actual cause of the symptoms and its context.
2. Determining type of therapy and goals to be achieved.
3. Setting up a hierarchy of anxiety-producing situations. In this hierarchy, the scariest
situations for the patient are placed at the top.
4. Exposure. Following the hierarchy, patients are gradually exposed to more fearful
situations. The patient remains in a specific situation until the anxiety has decreased to a
certain level.
5. Evaluation. The effectiveness of the therapy, usually measured against the predefined
goal, is evaluated and can initiate other psychological help.
3.6.1 Current In Vivo practice
In the daily practice of therapists, treatment may not be performed exactly according to
literature. To create a usable design, we need to understand the real problems that the users
are facing. Unfortunately, because of privacy reasons and patient confidentiality, it is not
possible to observe and record real therapy sessions. As an alternative, we have interviewed
therapists about their work.
Six therapists from five different institutions in the west of the Netherlands have been found
willing to participate in this study. A semidirected interview (Sebillote, 1995) was used, with
questions regarding the therapist’s experience with different phobias, work method, possible
problems with the current way of working and suggestions for the system to be designed.
The protocol for the interviews is provided in appendix A. The interviews took place in the
offices of the therapists and were recorded on tape for later analysis. The results of these
interviews have been summarized in the next paragraphs.
Patient population
In the interviews, the therapists reported that most patients have more than one problem. A
choice is made about which one of these problems will be solved during therapy. Of all
phobias, therapists encountered agoraphobia the most. Agoraphobia is defined as the fear of
being in places or situations from which escape might be difficult or embarrassing, such as
being in a crowd, traveling in an automobile, bus, plane or being in an elevator (American
Psychiatric Association, 1994).
Other phobias, such as claustrophobia and acrophobia, are quite common but people do not
often seek therapy for these phobias alone.
41

Chapter 3
Therapy process
The first part of the therapy is more or less standardized in the Netherlands. Patients, after
having been referred by their doctor or other institutes, partake in one or two intake-sessions.
During these sessions, the function analysis is performed and the patient’s background and
problems are examined. Some therapists use standard questionnaires during this phase,
others rely solely on interviews.
In a staff meeting, the conclusions of the intake-sessions are discussed and a course of
treatment is decided, along with the choice of which therapist is going to perform the
treatment. Often the choice of treatment is based on the availability of therapists and their
expertise.
There are several different types of treatment for phobias, the most common ones used are
behavioral therapy and cognitive therapy, or a combination of these two.
In behavioral therapy, the therapist and patient first determine a hierarchy of fearful
situations and a number of goals that should be achieved. Then the patient is exposed to
situations according to the hierarchy, where the patient is required to endure the situations
until the fear diminishes before proceeding to the next situations. Sometimes, the hierarchy
needs to be adjusted and sometimes the entire therapy must be revised.
In cognitive therapy the focus is on the cognitions that the patient has. An inventory of the
cognitions relevant to the phobia is made, such as ‘I’m going to faint’, ‘I’m going to fall’,
‘everybody is watching me’ or ‘I can’t breathe’. This can be done through interview or by
letting the patient keep a diary for some time. These cognitions are then discussed and
challenged by the therapist. For this the therapist often uses ‘behavioral tests’, where the
patient is exposed to the stimuli they fear to test whether their cognitions will come true. The
patient is required to stay in a situation until their fear diminishes before moving on to a
more difficult situation. In effect, both behavioral and cognitive therapy will be quite similar
during the actual exposure, as commented by several therapists: the patient will be
confronted with a fearful situation, and will move on to a more fearful situation once he or
she is no longer afraid in the current situation.
During exposure in both behavioral and cognitive treatment, several therapists instruct their
patients to report their fear on a scale from zero to ten. Two therapists also included
relaxation techniques into their therapy. Before the exposure therapy itself, the patient was
trained in these techniques and during exposure, the patient could use these techniques to
decrease the anxiety he or she was feeling.
Not all exposure is performed under direct guidance of the therapist. Sometimes the patients
have to perform 'homework', where they have to exposure themselves to anxiety provoking
situations. The next session, the patient will have to report their progress to the therapist.
Problems and suggestions
One of the biggest problems therapists are facing today is a lack of time, forcing them to do
less and less exposure in vivo because of required travel-time. Several therapists complained
about the lack of interaction with their patients during homework exposure.
3.7 Task Model
Based on the gathered data we can now form a model of the VRET process. We will use our
adaptation of the existing TA methods.
42

Analysis
3.7.1 High-level goals
At a sufficiently high abstraction level the task model is implementation independent. It is
therefore the same regardless of the VR system used or even whether VR is used at all. At
this level, our TM should be identical for both the current VRET situation and the In Vivo
exposure therapy situation. However, we will focus on that part of the therapy process that is
changed when VR is introduced: the exposure of the patient to the anxiety provoking stimuli.
We do not at this moment expect VR to change for instance the diagnosis or planning parts
of the therapy, even though perhaps with increased understanding of VR maybe one day VR
will be applied in these parts of the therapy process as well.
Figure 3.16 shows the high-level goals or tasks
of the therapist. Naturally, the main goal of the
therapist is to cure the patient. During exposure,
the therapist will change the fear a patient is
experiencing by manipulating the stimuli the
patient is exposed to and will determine whether
the patients fear is either too high or too low,
possibly changing the fear again accordingly.
Cure patient
Determine fear Change exposure Solve ambiguity
Figure 3.16: Therapist's goals
Also, during therapy, the therapist will respond to any questions the patient might have, thus
solving any ambiguity on the side of the patient. Even though this does not directly
contribute to curing the patient, it will facilitate the patient to perform his/her tasks.
Figure 3.17 shows the high level goals of the
patient during exposure treatment. The patient
will want to get rid of his or her fear. The
patient believes that this can be achieved in the
long run by following the therapist’s instructions,
thus curing the phobia. Unfortunately,
people with phobias will have a tendency to
avoid fearful situations and this tendency could
conflict with the therapist’s instructions.
Follow therapist
instructions
Get rid of of fear
Avoid fearful
situations
Figure 3.17: Goals of the patient
Solve ambiguity
Also, the patient will sometimes need to inquire about certain matters to resolve any
ambiguity about the therapy.
3.7.2 Media
An important part of the task implementation is the media available for communication.
Figure 3.18 shows an overview of the media available for the users in the VRET situation.
The therapist can control the VE by using either keyboard or joystick and can view the VE
on the screen. Furthermore, the computer system can generate sound. Both therapist and
patient can hear these sounds and can also produce sound themselves (i.e. talk). The patient
can see the images in the HMD and can change his or her viewpoint in the VE by changing
his or her body posture. We will define the posture as the location and orientation of the
patient’s body, and the configuration of the several parts of the body. Specifically the
movements of the patient’s head are monitored by the tracker. Here, we see a case of
Distributed Cognition: the changes in the posture of the patient are also directly observed by
the therapist and can be used to determine the patient’s experience.
43

Chapter 3
44
Computer system
therapist
controls
keyboard/
joystick
therapist
screen
therapist
therapist
screen
tracker
HMD
screens
speakers/
headphones
sound
HMD
sound
patient posture
sound
patient
patient
posture
Figure 3.18: Overview of the media used for
communication
3.7.3 Task decomposition
As shown in the preliminary analysis of the video-observations, there is not a clear
predetermined sequence of events in VRET. We will therefore use a representation of the
process that does not suggest such a sequence. Figures 3.19 to 3.24 show the decomposition
of the higher-level goals into the procedures encountered in VRET. These procedures or
lower level task components include the previously identified speech acts as well as the
‘observational acts’ that form the counterparts of these speech acts. These observational acts
form the second half of the interaction cycle, where the user observes, interprets and
evaluates the current state of the environment, and require state-information from the
environment through one of the available media. These information flows have also been
depicted at the bottom of the diagrams, indicating the media source providing that
information.
The decomposition does include some indication as to the sequence in which the events take
place, in a way similar to the HTA method. For every decomposition of a goal into its lower
level task components, we have used the following abbreviations:
• Seq.: Sequential. These tasks are usually performed in sequence.
• Sim.: Simultaneous. These tasks can be performed simultaneously.
• Alt.: Alternative: The user usually selects only one of these tasks at a time.
Figure 3.19 show the decomposition of the goal ‘Determine fear’ into two sub-tasks: The
speech act ‘Ask patient to report fear’, where the therapist verbalizes this request (through
the medium sound) and the observational act ‘Monitor patient response’. For this the
therapist monitors any utterances the patient makes (sound), and the movement of the patient
as seen on the screen or directly by observing the patient’s posture.

Goals
Procedures
Sound
T.Controls
T.Screen
P.Posture
Ask patient to to
to to report fear
Cure patient
Determine fear Change exposure
sim.
Monitor patient
response
Figure 3.19: Decomposition of the goal ‘Determine fear’
Solve patient
ambiguity
Analysis
45

Chapter 3
46
Cure Cure patient
patient
Determine fear Change Change exposure
exposure Solve ambiguity
Goals
alt.
Procedures
Instruct Instruct patient
patient Instruct Instruct computer
computer
sim.
seq.
Instruct Instruct computer
computer
to to move move patient
patient
alt.
Give Give instruction
instruction
alt.
Monitor Monitor Monitor patient patient
patient
position
position
Push Push joystick
joystick Press Press Press button button
button
Inform Inform patient
patient
about about VE VE change
change
Monitor Monitor patient
patient
Instruct Instruct patient
patient
to to move
move
Instruct Instruct patient
patient
to to change change posture
posture
Instruct Instruct patient
patient
to to look
look
Sound
T.Controls
T.Screen
P.Posture
Figure 3.20: Decomposition of the goal ‘Change exposure’

Analysis
Figure 3.20 shows the decomposition of the goal ‘Change exposure’. For this, the therapist
has two alternatives: either instruct the patient, or instruct the computer. Instructing the
patient consists of verbalizing an instruction and monitoring whether the patient executes the
instructions correctly. There are several different types of instructions that the therapist can
give, already identified as different speech acts. When instructing the computer, the therapist
usually follows a certain sequence: first, the therapist will inform the patient that he or she
will change the VE. Then, the therapist will instruct the computer, using either the joystick or
the keyboard, and monitor the effect of these actions on the computer screen.
Figure 3.21 shows the decomposition of the goal ‘Answer patient questions’. Here, we often
see a sequence where the therapist perceives a question from the patient and answers
accordingly. However, sometimes the therapist will also explain certain aspects of the
therapy or the VE without being asked to.
Cure patient
Goals
Determine fear Change exposure
Solve patient
ambiguity
Procedures
seq.
Sound
T.Controls
T.Screen
P.Posture
Monitor for
patient requests
Inform patient
about VE
Give answer
Figure 3.21: Decomposition of the goal ‘Solve ambiguity’
alt.
Inform patient
of of rationale
47

Chapter 3
Figure 3.22 shows the decomposition of the goal ‘Follow therapist instructions’. The patient
will listen for instructions by the therapist and attempt to execute these. The instruction can
be to either change the fear level the patient is currently experiencing by changing the
patient’s posture and possibly monitoring the effect this is having on the images perceived in
the HMD, or to verbally report the level of fear the patient is currently experiencing.
Goals
Procedures
Sound
HMD screens
P.Posture
seq.
Listen for therapist
instructions
Follow therapist
instructions
alt.
Change
experience
sim.
Get rid of of fear
Avoid fearful
situations
Execute therapist
instructions
Change posture Monitor VE
Solve ambiguity
Report fear
level
Figure 3.22: Decomposition of the goal ‘Follow therapist instructions’
Figure 3.23 show the decomposition of the goal ‘Avoid fearful situations’, or the tendency of
phobic people to avoid those situations that they fear. They will do this by changing their
posture such as clutching the railing, looking in another direction than where the fearful
stimuli is, or step back from a fearful situation. The results of these actions will in some
cases be visible in the VE. It is these often involuntary actions that can also betray the real
fear a patient is experiencing to the therapist.
Figure 3.24 show the decomposition of the goal ‘Solve ambiguity’. Here, the patient will
simply ask a question when something about the VE or the therapy is unclear and will wait
for a response from the therapist.
48

Goals
Procedures
Sound
HMD screens
P.Posture
Follow therapist
instructions
Get rid of of fear
Avoid fearful
situations
sim.
Change posture Monitor VE
Solve ambiguity
Figure 3.23: Decomposition of the goal ‘Avoid fearful
situations’
Goals
Procedures
Sound
HMD screens
P.Posture
Follow therapist
instructions
Get rid of of fear
Avoid fearful
situations
Ask question
Solve ambiguity
seq.
Listen for
response
Figure 3.24: Decomposition of the goal ‘Solve ambiguity’
Analysis
3.7.4 Functional requirements
The TA method used is similar in its hierarchical nature to many traditional and proven TA
methods. However, using the explicit distinction between goals and procedures makes clear
what goals are essential to the process and which goals are enabling goals: by-products of the
specific way of working. According to the decomposition, the functionality of the used
system for a phobia treatment application is sufficient, supporting each goal with at least one
procedure and at least one high level goal for each task.
49

Chapter 3
However, several tasks are not explicitly supported by the computer system but depend on
direct communication between therapist and patient. Possibly the effectiveness and
efficiency of the process could be improved if the computer could aid in these tasks. The
tasks that possibly provide the highest yield on improvement are those that are occur most
frequent. Our analysis has shown that the determining of the fear experienced by the patient
is the most frequent activity in VRET, a task that is hardly supported by the computer system
at all. Therefore, we should investigate if the task performance for these actions can be
improved with computer support.
3.7.5 Informational requirements
Apart from the functional requirements of the system, we can also investigate the
informational requirements of the system: does the system provide the users with the
information they need? Here we can use our distinction between state information and
procedural information. In appendix B you will find a complete overview of the information
that is provided and required during VRET per task. In this paragraph however, we will look
only at those flows of information that in some way do not completely fulfill the information
requirements.
State information
• Determining the patient’s fear
As discussed before, the process of determining the patient’s fear is a task that might be
improved with computer support. At this moment, the therapist uses several sources of
information to determine the patient’s fear, such as the patient’s posture and the
direction the patient is looking at, as visible on the therapist’s screen. However, for the
most time, the therapist relies on the use of SUDs to gain insight into what the patient is
experiencing. As several therapist’s commented, the absolute value of these SUDs have
no real meaning since each individual uses their own reference points in determining
this scale. Therapists will usually look for a change in SUDs, rather than a low or a high
SUD. Thus, to evaluate a SUD score reported by the patient, the therapist will also need
historical information about the SUDs reported by this patient. With the current system
configuration, the therapist is required to memorize this information during the therapy.
We see here a simple opportunity for improving the design of our system: we could
provide the therapist with cognitive artifacts representing the previous SUD scores.
• Collaborative steering
Other frequently occurring activities, that already are heavily supported by the
computer, are related to changing the view displayed in both the HMD and on the
therapist’s screen. The therapist can use either joystick or keyboard to change the
location of the patient in the VE to change the level of fear the patient is experiencing.
The patient can also change his/her position in the VE through use of the headtracker to
either follow the therapist’s instructions or to avoid fearful stimuli and thus reduce the
fear he or she is experiencing. Both therapist and patient will monitor the same images
generated by the computer for feedback on their actions. Here we see a problem of the
current way of working: If both therapist and patient are steering at the same time, it is
difficult to determine what changes in the VE are caused by whose actions. Also, the
actions by both users could counteract each other. Review of the video recordings
confirms that the users do have some problems with this. One problem encountered is
50

Analysis
that the patient keeps looking away from the direction the therapist wants to steer the
patient to. There is a problem of collaborative steering.
• View for navigation
Furthermore, the images generated by the system show only one viewpoint: the patient’s
viewpoint. This viewpoint is essentially used to expose the patient to the fearful stimuli
and shows only a small part of the VE at one time. When the therapist tries to use this
view to move the patient through the VE, some obstacles in the VE might not be visible,
making navigation difficult, especially in small spaces. Again, a review of the
recordings confirm that sometimes the therapist is attempting to move the patient
forward while in the VE the patient is colliding with an object not visible on the screen.
• Frames of reference
Another possible cause for usability
problems lies in the fact that the patient and
therapist use different types of input
devices. The patient has an input device that
uses the absolute location of the patient and
his or her posture in reality. Thus, there is a
limit to the locations that the patient can
specify with this input device because there
is a limit to this absolute space, such as the
range of the tracking device, the length of
the cables attached to the HMD and the area
of space available to move in. The therapist
however has an input device that is relative
in nature: when the therapist pushes the
joystick this will move the position of the
Virtual railing
Patient
Range of patient’s movement
Figure 3.25: Situation where the patient
is unable to look over the railing in the
VE.
patient in the VE relative to his or her current location. In effect, the therapist can
determine in what part of the VE the patient can move, and can sometimes inadvertently
prohibit the patient in performing certain actions. One example is that the therapist can
position the patient in such a situation where he or she cannot look over a railing, even
though the therapist might want him or her to do so. This situation is depicted in figure
3.25. We can say that the patient’s movement is limited in the real frame of reference,
and that the therapist has the ability to determine the mapping of this frame of reference
onto the frame of reference of the real world. However, with the current system, the
therapist needs to combine the information that can be extracted from viewing the
patient’s posture directly with the information that can be obtained from the screen to
evaluate this mapping. This could be difficult for the therapist, resulting in problems
when operating the system. The problem becomes even more serious when there are
objects in the real world that the patient can feel and that, at certain moments during
therapy, need to be co-located with a virtual object the patient can see in the HMD. In
our setup, the patient was surrounded by a railing, and one would like to align this
railing with a virtual railing, allowing the patient to grasp this railing and look over it as
depicted in figure 3.26. The video recordings show several instances where the therapist
is either requesting the patient to look over the railing while he or she is not able to, or is
trying to move a patient closer to a railing whilst the virtual railing is already aligned
51

Chapter 3
52
with the real railing. It is clear that we need
a way to help the therapist in creating a
correct mapping between the virtual and real
frame of reference.
Procedural information
Patient
Virtual railing
• Information from affordances
As described earlier, we would like our
system to have perceivable affordances that Real railing
hint at how the system can be operated. It Figure 3.26: Situation where the real and
seems that the patient can indeed extract virtual railing are aligned, allowing the
most procedural information from the patient to clutch the railing and physically
affordances of the system: the use of the look over it.
location of the patient’s head to determine
(with a one-on-one mapping) the patient’s viewpoint in the VE makes moving and
looking around in the VE very similar to what we humans are used to in reality.
However, for the therapist, there is some procedural information that can only be
obtained from manuals or training. This includes the relationship between the numbers
on the keyboard and the locations that the computer will move the patient to once these
numbers have been pressed. We can imagine that for more complicated VEs, where the
therapist has to alter several different elements of the VE during therapy using the
keyboard, this will become a major problem.
3.7.6 Usability problems in VRET
In chapter 2, we defined usability as consisting of the efficiency, learnability, memorability,
effectiveness or the number of errors and satisfaction. Based on our TA, we have found
several areas where the usability can be improved. In summary, these are:
• Fear determination
The process of determining the fear the patient is experiencing could possibly be
improved by increasing the computer support for this action. One possibility for this lies
in providing the user with cognitive artifacts that can be used to represent the historic
data of the patient’s SUD scores, allowing the therapist to work more efficiently.
• Collaborative steering
When both therapist and patient are steering, the current feedback to the users is
insufficient to separate the effect of each individual’s actions. Furthermore, the input
from both users can counteract each other, leading to navigation errors.
• Limited view for navigation
The limited view presented to both patient and therapist could be insufficient for
navigation, since obstacles will not always be visible, again leading to navigation errors.
• Frames of reference
The therapist needs insight into the mapping between the frame of reference of the
reality surrounding the patient and the frame of reference of the virtual world, to
understand where the patient can move him or herself and whether real objects are

Analysis
aligned with virtual ones. This should decrease the number of navigation errors the
therapist makes.
• Lack of affordances for the therapist
Currently, the therapist needs to memorize most procedures for operating the system.
Perceivable affordances should be introduces to increase the learnability and
memorability of the system.
Apart from this, we encountered two other usability problems during the experiment used as
our case study as described in paragraph 3.5.3.
• Lack of fear
Several patients did not experience any fear in the VEs despite their severe fear of
heights and this made therapy impossible. Therapists continued to attempt to change the
fear of the patient by moving him/her further in the VE, but based on the SUDs reported
by the patient, had to continue moving the patient until all available VEs were
completed within a single session. In other words: the effectiveness of these therapies
was low.
• Simulator sickness
Typical symptoms of simulator sickness are nausea, pallor, sweating, headache,
disorientation, dizziness, and residual effects. Simulator sickness, which is closely
related to motion sickness, can best be explained by the sensory-conflict theory (Cobb et
al., 1999). This theory proposes that the symptoms occur as a result of conflict between
signals perceived by the different senses. Several studies showed that simulator sickness
will increase during the exposure and will decrease over repeated exposures (Kennedy
et. al., 2000). Even though sessions were limited to two exposures of 20 minutes, several
patients reported feeling nauseous afterwards or even during the session. Clearly, this
causes problems with user satisfaction.
3.8 Conclusions
In this chapter we have proposed an adaptation and combination of existing TA methods to
analyze applications such as VRET. Applying this TA to our case showed that it is adapted
to some of the specific characteristics of applications such as VRET. Use of the Language /
Action Perspective allowed inclusion of both multi-modal HCI and human-to-human
interaction. Our analysis of the sequence of events showed that there is not a pre-determined
order of task, and thus the decision to not use a flow model seems validated. The use of a
task hierarchy proved to be useful in checking the functionality of the application, similar to
existing TA methods. Our focus on the implementation details in our hierarchy enabled us to
evaluate the information needs of the users, and compare these to the information provided
by the system through the various media. In general, we can say that the TA method
succeeded in revealing several usability problems, showing that the TA method does work, at
least to some extent.
Whether this TA method is fully applicable to VRET, and whether it actually is an
improvement over existing methods, should become clearer when this method is applied
further, not only identifying usability problems but also forming the basis for new designs.
53

Chapter 3
Several problems encountered during VRET did not follow directly from our TA, but might
be related to the HCI: The lack of fear in several patients and simulator sickness. The fact
that these problems did not follow from our TA is due to the fact that our TA method, similar
to other existing methods, only takes the user’s cognitions into account and not the user’s
non-cognitive processes, such as emotions and physiological reactions. Since these noncognitive
aspects play such an important role in applications such as VRET, we will discuss
them in great detail in the next chapter.
54

4 Presence
The research described in the brief history of VRET at the beginning of chapter 3 showed
that the use of VR has been found to be effective in treating phobias. However, our research
has also shown that there are several patients that do not respond to the VRET. To
understand whether we can improve the effectiveness of VRET, for instance by improving
the HCI, we must look at why VRET is effective in the first place. Unfortunately, very little
is clear about this subject.
It seems obvious that the effectiveness of therapy depends on the fact that the VR system
influences the user in a non-cognitive way by arousing the patient’s emotions such as fear. In
recent years, emotions have received more attention in the HCI literature. The term affective
computing has been used to denote the computing that relates to, arises from or deliberately
influences emotions (Picard, 1997). However, the main focus of this research has been, until
now, either on making the computer adaptive to the user’s moods (Crampes et
al., 2001), making the computer better at communication emotion to the user (Rivera et al.,
1994) or determining the effect of emotion on user behavior (Garnder & Meryl, 1985). No
research has yet properly addressed the way in which the computer can induce emotions in
the user.
In this chapter we will therefore investigate in what way the VR system effects the user.
According to literature (Hodges et al., 1994), the key concept that not only explains why
VRET is effective but can also can be called the defining aspect of VR in general is the
concept of presence. In the past few years a great deal of literature has appeared on the
subject, and this will be discussed in detail first 6 .
4.1 On the nature of presence
In presence research several definitions and theories have been proposed; the prominent ones
will be described here.
4.1.1 Definitions
The term "presence" is related to a wide field of research. Lombard & Ditton (Lombard &
Ditton, 1997) identified six different explications of presence that have been used in
literature: presence as
• social richness, the extent to which the medium is perceived as sociable, warm, sensitive
or personal when it is used to interact with other people.
• realism, the extent to which a medium can seem perceptual and/or socially realistic.
• transportation, the sensations of "you are there," "it is here," and/or "we are together".
• immersion, the extent to which the senses are engaged by the mediated environment.
• social actor within medium, the extent to which the user responds socially to a
representation of a person through a medium.
6 These paragraphs are based primarily on a previous publication (Schuemie et al, 2001)
55

Chapter 4
• medium as social actor, the extent to which the medium itself is perceived as a social
actor (e.g., treating computers as social entities) (Lombard & Ditton, 2000).
However, presence as discussed in literature related to immersive VR can most often be
characterized by the concept of presence as transportation: people are usually considered
"present" in an immersive VR when they report a sensation of being in the virtual world
("you are there"). The term co-presence or social presence is often reserved for the sense of
being together in a virtual world ("we are together").
Sheridan (Sheridan, 1992) makes another distinction. He emphasizes the difference between
presence, that is the sense of being in a computer-generated world, and telepresence, the
sense of being at a real remote location.
Heeter (Heeter, 1992) distinguishes between three different types of presence:
• Personal presence, a measure of the extent to which the person feels like he/she is part
of the VE.
• Social presence, refers to the extent to which other beings (living or synthetic) also exist
in the VE.
• Environmental presence, refers to the extent to which the environment itself
acknowledges and reacts to the person in the VE.
Schloerb (Schloerb, 1995) distinguishes two types of presence:
• Subjective presence, the likelihood that the person judges himself to be physically
present in the remote or virtual environment.
• Objective presence, the likelihood of successfully completing a task.
Schloerb's definitions of subjective and objective presence represent a completely empirical
point of view. Schloerb questions the value of subjective presence, because objective
presence, the ability to work, should be the most important criteria for a VE.
An important distinction proposed by Slater & Wilbur (Slater & Wilbur, 1997) is that
between "presence" and "immersion".
• Immersion, an objective description of aspects of the system such as field of view and
display resolution.
• Presence, a subjective phenomenon such as the sensation of being in a VE.
A less often used but often cited taxonomy is that of Zeltzer (Zeltzer, 1992) who argues that
a VR system can be characterized by its:
• Autonomy: extent to which the VE is more than just passive geometry.
• Interaction: degree to which VE parameters can be modified at runtime.
• Presence: measure for the number and fidelity of available sensory input and output
channels.
Here, Zeltzer uses the term "presence" in a way that closely resembles the term "immersion"
defined by Slater. Unfortunately, the word "immersion" is sometimes also used in a way
closely resembling the subjective definition of presence (e.g. Bangay & Preston, 1998 and
56

Presence
Witmer & Singer 1998). In this dissertation Slater & Wilbur's definitions of the terms
"immersion" and "presence" will be used, unless stated otherwise.
4.1.2 Theories
In the literature, several theories on the nature of presence in immersive VR have been
proposed. These will be described below.
Presence as non-mediation
In a discussion on the mailinglist 'Presence-L Listserv' during the spring of 2000 a general
explication of presence was defined, which is still being refined today. The latest version to
date is:
Presence (a shortened version of the term "telepresence") is a psychological state or
subjective perception in which even though part or all of an individual's current experience
is generated by and/or filtered through human-made technology, part or all of the
individual's perception fails to accurately acknowledge the role of the technology in the
experience. Except in the most extreme cases, the individual can indicate correctly that s/he
is using the technology, but at *some level* and to *some degree*, her/his perceptions
overlook that knowledge and objects, events, entities, and environments are perceived as if
the technology was not involved in the experience (Lombard, 2000)
(Experience is defined as a person's observation of and/or interaction with objects, entities,
and/or events in her/his environment; Perception, the result of perceiving, is defined as a
meaningful interpretation of experience.)
This explication emphasizes a dualism: part of the perception acknowledges that the
experience is mediated by technology, while another part does not. This is rarely explicitly
stated in presence literature even though it is an essential aspect of the concept. People
always know the experience is mediated and, given the current state of technology, can
always distinguish between mediated and direct stimuli. Nevertheless, at some level, the
illusion of non-mediation can be perceived. This means that even though patients know they
are not really in the situation they fear they will respond as if they were.
Presence as focusing of attention
(Witmer & Singer, 1998) relate presence in part to the concept of attention: "presence may
vary across a range of values that depends in part on the allocation of attentional resources."
• Involvement, a psychological state experienced as a consequence of focusing one's
attention on a coherent set of stimuli or related activities and events.
• Immersion, defined as a psychological state characterized by perceiving oneself to be
enveloped by, included in, and interacting with a VE. (Note that this definition of
immersion is quite different from the often-used definition of immersion as an objective
measure as used in this thesis.)
Both involvement and immersion are thought to be necessary for experiencing presence. The
authors state that by focusing attention a person will get more involved and will as a
consequence experience a higher sense of presence. They call presence similar to the concept
57

Chapter 4
of selective attention which “refers to the tendency to focus on selected information that is
meaningful and of particular interest to the individual.”
(Slater et al., 1994) stress the participant's sense of "being there" in the virtual environment,
and point out that a high sense of presence in a VE requires a simultaneous low level of
presence in the real world and vice versa. (Biocca, 1997) states that "at one point in time,
users can be said to feel as if they are physically present in only one of three places: the
physical environment, the virtual environment, or the imaginal environment. Presence
oscillates among these three poles." The level of presence experienced during an interval is
dependent on the relative amount of time being present in the virtual world (Slater & Steed,
2000). This idea of ‘exclusive presence’ is closely related to the attention of users that can be
either focused on the VE or on the real or imaginary world.
Presence as natural interaction
Another view of presence is based on the ecological theory of perception. (Flach & Holden,
1998; Schuemie & van der Mast, 1999; Zahorik & Jenison 1998) Basic concepts of this
approach are:
•
•
•
58
The environment offers situated affordances. The term affordance was coined by Gibson
(Gibson, 1979) and is meant to describe the possibilities or opportunities that the
environment offers for a particular organism.
Perception-action coupling. An organism perceives its environment primarily in terms
of its affordances, making perception dependent on possible action.
Tools become “ready-to-hand.” According to Heidegger, using a tool eventually stops
the user from possessing a stable representation of the tool. The user is no longer aware
of the tool itself but only of the usefulness the tool has in whatever task is performed
(Zahorik & Jenison 1998). For example, many people who write with a pen no longer
think of the pen as an object that they’re holding in their hands, but are thinking only of
the tip of the pen and the mark it makes on the paper.
Applying the concept of perception-action coupling to VR one can conclude that the user
will perceive the VR equipment in terms of what can be done with it, i.e. interacting in the
VE. The mediating technology itself will eventually become “ready-to-hand”, invisible to the
user, thus creating a sense of presence where the illusion of non-mediation is perceived.
(Zahorik & Jenison, 1998) state that successfully supported actions in an environment will
lead one to perceive oneself as existing in that environment, to a sense of presence. An action
is said to be successfully supported when the result of that action is considered lawful:
responses from the environment must be similar to those in the real-world environment in
which our perceptual system evolved.
Other researchers agree with the key role of perceived possible interactions in presence but
stress that "action is essentially social" (Mantovani & Riva, 1999) Experiencing presence
depends on whether the VE behaves and is constructed according to our cultural expectations
and whether the VE is perceived and interpreted the same by others in the VE (O'Brien et al.,
1998)
(Sheridan, 1999) proposes the estimation theory, stating that users will create a mental model
which estimates reality, based on our senses and interaction with that reality. This model is

Presence
used to understand and predict the effect of our interactions and plays an important role in
the sense of presence.
In a similar vein, (Schubert et al., 1999b) propose the embodied cognition framework by
Glenberg as a means for explaining presence. A mental representation of the environment is
made in terms of patterns of possible actions, based on perception and memory. "Presence is
experienced when these actions include the perceived possibility to navigate and move the
own body in the VE."
In predicting the outcome of actions, humans have the ability to suppress contributions of the
current environment to conceptualization, thus explaining why we can experience presence
in a VE despite sensing conflicting features of the real environment.
4.1.3 Conclusions on the nature of presence
The previous paragraph listed several definitions and theories on the nature of presence that
do not necessarily contradict each other, although they can have different implications. To
refine the concept of presence, better instruments for measuring presence are needed.
Unfortunately, the way presence is measured depends on the theory used. (Prothero et al.,
1995b) suggest that one way to escape this circularity is to use a converging approach: based
on a simple theory, a measure can be developed which can be used to improve the theory,
and so on. Already several research groups attempted such an approach, as will be shown in
the section on measures.
(Kalawsky, 2000) warns that "presence is a multi-dimensional parameter that is arguably an
umbrella term for many inter-related perceptual and psychological factors." Most theories
mentioned above attempt to explain presence in terms of several underlying factors. One
common factor is that of attention. Also, most scholars relate presence to a mental model of
the virtual reality with the (virtual) body in it.
In fact, (Prothero et al., 1995b) claim that " 'Presence' and 'situation awareness' are
overlapping constructs." (Lackner & Dizio, 1998) supply anecdotal evidence that when
people have difficulty forming a mental model of a real (but nonterrestrial) space, they also
report a loss of sense of presence.
Most theories and definitions of presence mentioned above attribute to presence the fact that
the human mind, in part or completely, experiences a VE as if it were real. Presence is
therefore essential for exposure therapy, since this type of treatment depends explicitly on
confronting patients with the situations they fear. If the patient does not feel present in the
anxiety-provoking situation, he or she will not become afraid and habituation will not occur.
4.2 Impact of presence
For the concept of presence to be useful and be applicable in practical situations it is
important to understand the impact or consequences of presence. What types of phenomena
have been linked to presence? Does it really lead to patients responding to virtual fearful
stimuli as if they were real, and what other consequences can a high sense of presence have?
This section will review the theories and empirical studies on the usefulness of presence, and
thus its relationship to other constructs.
4.2.1 Subjective sensation
Almost every theory on presence refers to the subjective sensation of "being there"
experienced and reported during immersion in a VE and this sensation is in fact part of most
59

Chapter 4
definitions of presence. However, for instance the explication statement from the Presence-L
Listserv (Lombard, 2000) does not exclude a state where one does not have an explicit
sensation of "being there" but still could be said to experience some form of presence.
This subjective sensation may apply to the environment currently being experienced, or to
memories of past experiences. As (Slater et al., 1999) note, a key result of presence is that a
person remembers the VE as a place rather than a set of pictures.
4.2.2 Task performance
As (Welch, 1999) stated, "there is a pervasive belief that presence is causally related to
performance … Despite the popularity of this notion, however, there is no solid evidence to
support it." Welch continues by quoting Witmer in a personal communication about the
review of several research projects: "significant correlation between presence and
performance were the exception rather than the rule." (Ellis, 1996) even argues that for some
tasks, less presence might lead to better performance, for instance when a more abstract view
of an environment is more helpful for completing the task.
(Mania & Chalmers, 2000) confirm that presence need not be related to task performance in
an empirical study with three conditions: lectures were given on a specific topic in the real
world, in a virtual classroom and an audio only environment. In a between-subject design 18
subjects were assigned to each condition. Presence was found not to be correlated with the
task performance of acquiring knowledge during the lecture.
(Kim & Biocca, 1997) however, in a study involving 96 subject being exposed to an
infomercial on TV, did find a weak but significant correlation between the part of their
presence questionnaire labeled "departure" and both factual memory and average recognition
speed for recognizing stills from the infomercial.
(Slater et al., 1998a) and (Steed et al., 1999) found a relationship between immersion and
leadership. In a between-subject design with 30 subjects and three conditions (HMD and 2
types of desktop-VR systems), where subjects had to solve a riddle together, those in the
more immersive conditions tended to evolve as the leader of the group. However, no
relationship between immersion and presence was found in this study.
4.2.3 Responses and emotions
Perhaps one of the most important consequences of presence is that a virtual experience can
evoke the same reactions and emotions as a real experience.
(North et al., 1998) found that people can show signs of fear of public speaking when
confronted with a virtual audience. (Slater et al., 1999) in a between-subject study with ten
subjects and two conditions (positive and negative audience) showed that, in a regression
analysis, presence tended to amplify subject's response to the audience. In other words:
People experiencing a higher level of presence were prone to report more negative reactions
to a negative audience and more positive reactions to a positive audience.
When confronted with visual cues suggesting motion, a person will tend to correct for the
perceived motion by adjusting their body posture. (Freeman et al., 2000), in an experiment
with 24 subjects, investigated the relationship between reported presence and postural
responses. No significant correlation (r = 0.025) was found however.
The most striking examples of how presence can lead to emotional responses similar to the
real world are the success stories of phobia treatment. During phobia treatment, patients
respond in similar fashion to the virtual fearful situations as they would to the real situations.
Reported responses include increased sweating etc. (Hodges et al., 1994)
60

Presence
(Regenbrecht et al., 1998) investigated the relationship between presence and fear of heights,
both measured by questionnaires. In an experiment with 37 non-phobic subjects they did not
find a significant correlation (r = 0.251, p>.10) between presence and fear. A regression
analysis did show that presence was the best predictor of fear. (Schuemie et al., 2000), in an
explorative study with ten subjects being treated for fear of heights, did find a significant
correlation between fear and presence reported on questionnaires (r=0.4461), but no
significant correlation (r = -0.325) between presence and reduction of acrophobia (also
measured through questionnaires).
(Krijn et al, in preparation) compared the HMD-based system described in the previous
chapter to a more advanced CAVE system for treatment of phobias. 37 subjects participated
in the experiment. Although they reported higher levels of presence in the CAVE condition
(F=8.024, p=.010) ,they did not report higher levels of fear, nor was there any significant
difference in the effectiveness of the therapy in either conditions. However, in this study
there was a large amount of dropouts (12), mostly because the subjects did not experience
enough fear. In the HMD condition, there were seven dropouts compared to the three
dropouts in the CAVE condition that were attributable to low presence.
A chart-review of the VR therapies already given at the Center for Advanced Multimedia
Psychotherapy showed that the main reason behind the number of dropouts was the lack of
presence in these subjects (Presentation by Mark Wiederhold at the MMVR2002 conference,
Newport Beach, California)
In-depth investigation
In an attempt to clearly determine the relationship between presence and fear of heights, we
performed a small preliminary study in collaboration with the University of Amsterdam
(Krijn et al., 2000). In this study 30 subjects participated, 15 of which suffered from fear of
heights. Subjects were exposed to all four conditions in a random order in a two by two
within-subject design. The two factors manipulated were the VE, which was either a height
situation or a neutral situation, and the level of immersion. In the low-immersion condition,
subjects could not move the viewpoint at all but instead watched a predetermined path
through the VE. In the high-immersion condition, subject could rotate the viewpoint through
headtracking and could move forward in the VE by pressing a button on a handheld device.
Subjects were furthermore given the instruction to locate small cubes in the VE, which lit up
when approached near enough.
Results showed that presence was much higher in the high-immersion conditions (t=-2.96,
p=0.006), but fear did not differ significantly between conditions. One possible explanation
for this could be the fact that in the low-immersion condition, subjects could not avoid
looking down into the depth, whilst in the high-immersion condition subjects could avoid the
edge altogether and, according to observations, did so in numerous occasions.
4.2.4 Simulator sickness
One problem associated with using VR is that it can cause nausea and dizziness, a
phenomenon known as "simulator sickness". (Witmer & Singer, 1998) found a significant
negative correlation between simulator sickness reported on the Simulator Sickness
Questionnaire and presence measured using the Witmer & Singer questionnaire (r = -0.426,
p

Chapter 4
4.2.5 Conclusions on results of presence
Based on the current status of presence research much uncertainty remains as to the results of
presence. Presence, when defined as a subjective sensation, can be a goal in itself for certain
applications such as games and movies. Whether presence can contribute to better task
performance is controversial based on the reported findings.
And although the usefulness of presence for emotional responses and phobia treatment seems
to have more empiric evidence, this too is inconclusive. One problem here is that only weak
evidence for a relationship between presence and emotional responses such as fear has been
found, but no study has yet addressed the causality of this relationship. In other words, it is
still unclear whether higher measured presence causes stronger emotional responses in a VE
or the other way around. Our small experiment aimed at determining the relationship
between presence and fear of heights showed that when HCI has an effect on presence, that
this increase in presence does not necessarily lead to more fear. It seems that HCI can also
have another effect on fear, when the HCI enables the user to avoid fearful situation.
The study by (Krijn et al., in preparation) comparing the use of an HMD to the use of a
CAVE for VRET showed that although presence was higher in the CAVE condition, this did
not lead to higher fear or greater therapy effectiveness. However, one should keep in mind
that during therapy, the fear of the patient is in fact manipulated by the therapist, who will
not allow the fear to become too high or too low. It is only when the fear cannot become high
enough that we can expect a relationship between presence and fear and between presence
and therapy effectiveness. In other words, we should expect a relationship between level of
presence and the number of failures of therapy. In the aforementioned study, there were more
dropouts in the HMD condition than in the HMD condition. In the chart review (ref. chart
review Mark Wiederhold), failure of VRET was mainly attributed to an insufficient sense of
presence.
4.3 Measuring presence
Knowing what phenomena presence can result in allows us to create a measure for presence.
A distinction can be made between subjective measures, requiring introspection by the
subjects, and objective measures. Objective measures can further be divided into behavioral
and physiological measures. These types of measurement will be reviewed below.
4.3.1 Subjective measures: Questionnaires
The most commonly used methods to measure presence in current research are based on
subjective ratings through questionnaires.
As (Witmer & Singer, 1998) rightly stress, any measure of presence should be both reliable,
i.e. only dependent on the characteristics under consideration, and valid, i.e. measuring what
it intends to measure and measuring it well. More specifically, the authors emphasize that a
measure or scale can have content validity or “the coverage of the measured behavioral
domain by the scale items” and construct validity, “the extent to which it can be said to
measure a theoretical construct or trait.” If available, these characteristics of the
measurements will be mentioned here as well.
An advantage of questionnaires is that not only subjective sensations during the experience
in a VE can be measured. Subjects can also be asked to describe the VE and their own
physiological and behavioral responses, although these observations are of course less
reliable because of their subjective nature.
62

Presence
Sometimes these questionnaires consist of a single question, e.g. "I feel a sense of actually
being in the same room with others when I am connected to a MOO" (Towell & Towell,
1997). To make the measure more reliable often several questions are used (Dinh et al.,
1999; Hendrix & Barfield, 1996a). Recently a more or less systematic approach has been
taken to establish reliable and validated questionnaires. The most prominent ones will be
described here.
Slater and colleagues
The questionnaire developed by Slater and colleagues evolved over a number of studies
(Usoh et al., 2000) and has received much attention in presence research. It is based on
several questions, which are all variations on three themes: (Slater et al.,1995)
• The subject’s sense of “being there”.
• The extent to which the VE becomes more “real or present” than actual reality.
• The “locality,” the extent to which the VE is thought of as a “place” that was visited
rather than just a set of images.
These themes are directly derived from the research group's theory on the nature of presence
and are all strictly related to results of presence. The presence score is taken as the number
of answers that have a high score.
Witmer & Singer
Based on their theory of involvement and immersion and on previous empirical and
theoretical research, (Witmer & Singer, 1998) determined several factors that are thought to
contribute to a sense of presence:
• Control factors, the amount of control the user had on events in the VE.
• Sensory factors, the quality, number and consistency of displays.
• Distraction factors, the degree of distraction by objects and events in the real world.
• Realism factors, the degree of realism of the portrayed VE.
On their Presence Questionnaire (PQ), users can rate their experience in the VE according to
these factors on questions with a 7-point Likert scale. The presence score is the sum of these
ratings.
To validate this questionnaire, 152 subjects were asked to answer the questions after using a
VE. The correlation between single items and the total score was investigated and most items
showed a strong correlation. Items showing no significant correlation were deleted.
However, as (Slater, 1999) pointed out, no correction was made for the fact that the item
score already would be correlated with the total because the item itself is included in the
sum. It is also interesting to note that the PQ is attempting to measure presence by measuring
its causes, as evaluated by the user, and not its results.
The PQ was reduced by dropping items that did not contribute to its reliability. From a
cluster analysis on data from this reduced PQ scale Witmer & Singer found three factors,
which did not perfectly match the original factors, mentioned above. These factors, which
regrouped items from the original factors, were labeled:
•
Involved/Control, the control and responsiveness of a VE, and how involving a VE is.
63

Chapter 4
•
•
64
Natural, the naturalness of interactions and control of locomotion, and the consistency
of a VE.
Interface Quality, the amount of interference or distraction from task performance, and
the participant’s ability to concentrate on the tasks.
Calculations over multiple experiments indicated the reliability of the PQ to be 0.88
(Cronbach’s α, N = 152). On the subject of content validity, the authors state that the PQ
items were based on a review of the presence literature and “tap both aspects of presence:
involvement and immersion.” Witmer & Singer did a preliminary construct validation by
checking the association with other variables and constructs such as simulator sickness, task
performance, natural modes of interaction, spatial ability tests and relation to their Immersive
Tendency Questionnaire measuring the tendency to become involved or immersed. The
results of this validation were positive.
Igroup Presence Questionnaire (IPQ)
(Schubert et al., 1999a) constructed their IPQ by combining previous published
questionnaires, among which those of Witmer & Singer and Slater & colleagues, with a
questionnaire from earlier research (Regenbrecht et al.,1998) and some newly developed
questions on technological and context variables. The resulting 75-item questionnaire was
submitted to 246 volunteers, most of who were male desktop-based-VR users. It should be
noted that most entries were from game-based VR systems, and only a small minority
involved a HMD or a CAVE system. From a factor analysis eight factors were extracted;
three of these were found to be concerned with presence itself, and five were identified as
immersion factors. The presence factors, which entailed only subjective reports of how users
experienced the VE, were:
• Spatial presence (SP), the relation between the VE as a space and the own body.
• Involvement (INV), the awareness devoted to the VE.
• Realness (REAL), the sense of reality attributed to the VE.
The immersion factors, the factors concerned with descriptions of the interaction of the user
with the VE or with descriptions of the technological side of the VE, were:
• Quality of immersion (QI), the sensory quality for richness and consistency of the
multimodal presentation.
• Drama (DRAMA), the perception of dramatic content and structures.
• Interface awareness (IA), the awareness of interfaces that distracts from the VE
experience.
• Exploration of VE (EXPL), the possibility to explore and actively search the VE.
• Predictability (PRED), the ability to predict and anticipate what will happen next.
The authors contend that the factor analysis provides supporting evidence for a distinction
between reports on subjective experiences - the presence factors - and reported evaluations of
the technology - the immersion factors. Furthermore, the two factors spatial presence (SP)
and involvement (INV) support the distinction between a spatial-constructive and an
attention component. This distinction was postulated earlier by (Witmer & Singer, 1998) and
was also derived by the authors from the Embodied Presence Model. Finally, (Schubert et

Presence
al., 1999a) state that the two factors SP and INV together load on a first second-order factor,
which thus might be a general presence factor. Though not predicted by the model and to the
authors’ surprise a third factor "realness" also loaded on this general presence factor.
Schubert & colleagues calculated the internal consistency of IPQ over two studies to be 0.85
and 0.87 (Cronbach’s α; N = 264, N = 296).
Kim & Biocca
(Kim & Biocca, 1997) constructed a questionnaire with 8 items based on theory and
questionnaires by other authors. 96 subjects filled in the questions after having been exposed
to an infomercial on a TV. A factor analysis found 2 factors, which were labeled using a
metaphor of transportation:
• Arrival, being present in the mediated environment.
• Departure, not being present in the unmediated environment.
The arrival factor included items related to feeling one had arrived in another world than the
real one (the world of the infomercial); the departure factor included items relating to the
feeling one had never left the real world.
ITC Sense Of Presence Inventory (ITC-SOPI)
The questionnaires mentioned so far were designed with certain media in mind, such as
immersive VR. (Lessiter et al., 2000) attempted to create a measure that should apply across
a range of media, for example also to television and cinema. For their ITC-SOPI
questionnaire 63 items were generated which were thought to have relevance to the concept
of presence. All items had a 5-point Likert scale. The questionnaire was administered to 604
people following their experience of a mediated environment. Factor analysis found 4
factors:
•
•
•
•
Physical Space, e.g., "I had a sense of being in the scenes displayed", "I felt I was
visiting the places in the displayed environment", "I felt that the characters and/or
objects could almost touch me."
Engagement, e.g., " I felt involved (in the displayed environment)", "I enjoyed myself",
"My experience was intense."
Naturalness, e.g., "The content seemed believable to me", "I had a strong sense that the
characters and objects were solid", "The displayed environment seemed natural."
Negative effects, e.g., "I felt dizzy", "I felt disorientated", "I felt nauseous."
Eventually 44 items were retained which loaded on one of these factors. The similarity
between the first three factors and the factors identified by (Schubert et al., 1999a) is, as
noted by the authors, striking. Lessiter & colleagues calculated the internal consistency
coefficients for each of the four factors and found alphas ranging from 0.94 (Physical Space)
to 0.76 (Naturalness). The authors did a preliminary validation of their questionnaire by
comparing its results to results they obtained by using comparable (but not identical)
questions from the Slater et al. questionnaire. The results showed that both questionnaires
load onto the same factors and can discriminate between different media. Further studies are
planned to directly compare the ITC-SOPI with Slater et al.’s questions. The authors also
computed scale scores for each factor for the different media across which data were
65

Chapter 4
collected. They found that the scores for the factors were correlated to the media format in a
predictable way. For instance, the factor Physical Space showed sensitivity to media format.
Lombard & Ditton
(Lombard & Ditton, 2000) are also creating a cross-media presence questionnaire.
Participants were assigned to one of two conditions: high or low presence. The high presence
condition involved a 3D IMAX movie, the low presence condition a 12-inch black and white
television. A 103-item questionnaire was developed based on items used by other authors. So
far, 307 subjects completed the high presence condition. The experiment is still ongoing. In a
factor analysis of the preliminary results the following factors are found:
• Immersion, relates to the sense of immersion, involvement and engagement in the
mediated environment.
• Parasocial interaction, relates to interacting with other people in real time in the
mediated environment.
• Parasocial relationships, concerns feelings of friendship etc. towards people in the VE.
• Physiological response, concerns amongst others simulator sickness.
• Social reality, relates to how likely the events are to occur in reality.
• Interpersonal social richness, relates to how well user could observe interpersonal
communication cues.
• General social richness, relates to items such as unemotional / emotional, unresponsive
/ responsive, impersonal / personal.
Conclusions on questionnaires
Presence questionnaires are all originally developed from certain theoretical views on the
concept of presence. This basis determines the scope of the questionnaires as well as the
relevant application domain, such as immersive VR or TV.
In turn, the measures are used to refine the theories on which they are based. Techniques
such as factor analysis show that there are several major components in reported subjective
sensations on presence, and that these components are related. However, the relationships
between these components and the concept of presence may not be assumed as given, and
depends in part on the definition of presence used. Also, the components found depend very
much on the original scope of the study and its related questionnaire. This is obvious for
instance in the non-interactive design of Lombard’s experiment. Another example is the Kim
& Biocca questionnaire with its limited number and diversity of questions, which therefore
was limited in the number and diversity of factors found.
Subjective measures can be prone to errors. For instance, prior experience in rating stimuli
on other aspects such as interest and 3D-ness affected the subsequent rating of presence in a
study involving 72 subjects (Freeman et al., 1999). Similar results were found by (Welch et
al., 1996) in a preliminary study, where the order in which the VEs were presented was
found to have a main effect on the relative presence ratings of these VEs when compared to
each other.
(Usoh et al., 2000) subjected two questionnaires to a reality-test in a between subject design;
ten subjects completed a search task in reality, ten completed the task in a VE. If the
questionnaires measured correctly, subjects should report a greater sense of presence in
reality than in the VE. However, subjects did not report significantly different presence
ratings on the Witmer & Singer PQ for the two conditions. The questionnaire by Slater &
66

Presence
colleagues did find a marginal but significant difference. Interestingly, (Mania & Chalmers,
2000) when using the latter questionnaire, found presence to be "much higher and
significantly different" for a real situation compared to two virtual ones (18 subjects in the
real condition, 18 subjects in an immersive VE and 18 subjects in an audio-only VE).
4.3.2 Other subjective measures
Continuous measure
Instead of administering a questionnaire only after a virtual experience, (IJsselstein & de
Ridder, 1998) proposed a continuous measure of presence during the experience. In a study
involving 24 subjects, a hand-operated slider could be used to indicate the level of presence
experienced at that moment. Analysis showed that rated presence increased with the addition
of stereoscopy and of motion cues.
Presence Counter
Based on their theory that individuals are, at one moment in time, either completely present
in the real or in the virtual world, (Slater & Steed, 2000) proposed a presence counter which
measures the number of transitions in presence. Because the action of reporting such a
transition requires a person to feel present in the real world, only transitions from the virtual
to the real world can be recorded. Based on a simple Markov Chain model, this counter is
used to estimate the relative time the person was present in the virtual world. In a betweensubject
study with 18 participants significant correlations were found between body
movement and presence (r 2 =0.38) and between presence measured through the presence
counter and through a traditional questionnaire (r 2 =0.32).
Focus Group Exploration
To gain more qualitative insight into the concept of presence, (Freeman & Avons, 2000)
used Focus Group Exploration. This method requires small groups to discuss a topic, in this
case people's experience while watching stereoscopic TV. Results show that non-experts
describe sensations of presence, and are relating presence to involvement, realism and
naturalness. (Heeter, 1992) applied a similar approach, questioning users after they had used
immersive VEs.
4.3.3 Objective measures: Behavioral
As mentioned in the section on results of presence, people tend to respond to mediated
stimuli as if it were unmediated when they experience a high level of presence. Examining
people's reaction to mediated stimuli could provide an objective measure of presence.
(Sheridan, 1996) proposes measuring reflex responses, such as automatically trying to catch
a ball or trying to avoid a rapidly approaching object. As mentioned before, (Freeman et al.,
2000) attempted to use postural response as a measure for presence but found no significant
correlation between this measure and reported presence.
(Prothero et al., 1995b) propose so-called "class A" measures of presence, which measure
subjects’ responses to virtual cues when subjects are also presented with conflicting real
cues. (Slater et al., 1995) used such a method: subjects were shown a radio in reality and then
had to put on a HMD which showed a VE with a radio at the same location as the real one.
During the experiment, the real radio was moved and turned on. The subject was required to
67

Chapter 4
point at the radio when this happened. A high degree of presence would lead to the subject
pointing at the virtual radio rather than the real one. In the study, involving 8 subjects, a
significant correlation was found between this objective measure and a presence
questionnaire.
(O'Brien et al., 19998) used an ethnographic approach to study user's behavior, analyzing
human-to-human interaction in multi-user VEs. This lead to more qualitative insight in the
nature of presence, linking presence to the concept of intersubjectivity, i.e. the things people
know in common.
4.3.4 Objective measures: Physiological
(Sheridan, 1992) warns that “ ‘Presence’ is a subjective sensation, much like ‘mental
workload’ and ‘mental model’ – it is a mental manifestation, not so amenable to objective
physiological definition and measurement.” (Meehan, 2000) however, attempted to measure
presence using heart rate, skin temperature and skin conductance in an experiment where 10
subjects were exposed to a virtual height situation in a sophisticated immersive VE.
Although results regarding heart rate and skin temperature were inconclusive due to noise in
the heart rate measurement and slow change in skin temperature, a correlation was found
between skin conductance and presence as measured using the questionnaire by Slater &
colleagues. These results tend to be supported by the findings of (Wiederhold et al., 1998),
who performed a within-subject experiment with five subjects, one diagnosed as having fear
of flying. During exposure to an airplane simulator on either a screen or using a HMD, skin
conductance was found to be significantly higher for the HMD condition, which also
generated the highest presence ratings on a presence questionnaire.
It is important to note that skin conductance, like most physiological measures, is related to
arousal and not directly to presence. In the two experiments mentioned above, higher
presence was related to a decrease in skin conductance level, indicating increased arousal.
For illustration: (Wilson & Sasse, 2000) report that at lower video frame rates skin
conductance is lower, indicating stress and arousal. Lower frame rates are however
associated with lower presence as described in the section on causes of presence.
4.3.5 Conclusions on measures
Measuring presence is done almost exclusively via questionnaires, using them to refine the
theories on presence and, surprisingly, to validate objective measures. A reason for this is the
fact that presence theory is still being developed, and questionnaires offer rich feedback
required to aid in the understanding of the phenomenon being measured.
For our own experiments, described in the following chapters in this dissertation, we will
need to choose one of the measures. It is clear that questionnaires are by far the bestdeveloped
measures. Our preference goes out to the IPQ for the following reasons:
• As described before, the PQ by Witmer & Singer measures presence by measuring its
causes. It assumes the relationship between these causes and presence itself to be
known. However, it is exactly this relationship that is the subject of our research.
• The presence questionnaire by Slater et al. has been used many times, but nevertheless a
careful validation has never taken place.
• The other questionnaires are less specific to immersive VR and often apply more to
other media such as television.
68

Presence
The Igroup Presence Questionnaire (IPQ), originally consisting of 75 items, has been tested
as described earlier and the number of items have been reduced to 14, covering three factors
labeled ‘Spatial presence’, ‘Involvement’ and ‘Realness’.
4.4 Causes of presence
If presence does indeed play an essential role in VRET, we must understand what factors can
contribute to a sense of presence. Already, much research has been devoted to finding these
factors. In fact, several researchers constructed different categorizations of these factors.
(Slater & Usoh, 1993):
• High quality, high-resolution information.
•
•
•
•
•
•
•
•
•
•
•
•
Consistency across all displays.
Interaction with environment.
virtual body, the representation of the user's body in the VE.
Effect of action should be anticipated..
(Witmer & Singer, 1998):
Control factors, the control the user has.
Sensory factors, the richness of the displayed information and consistency across
displays.
• Distraction factors, how much the user is distracted from the VE.
• Realism factors, pictorial and social realism of the VE.
(Sheridan, 1992):
Extent of sensory information.
Control of relation of sensors to environment.
Ability to modify physical environment.
(Lombard and Ditton, 1997)
The form in which the information is presented.
The Content of the information.
User characteristics.
(Steuer, 1992):
• Vividness refers to the ability of a technology to produce a sensorial rich mediated
environment.
• Interactivity refers to the degree to which users of a medium can influence the form or
content of the mediated environment.
• User characteristics refers to the individual differences in users.
In the following paragraphs an overview is provided of the empirical research concerning
these factors. For this a categorization is used closely resembling that of Steuer. This choice
was made based on the following argument: first of all it is important to make a distinction
between characteristics of the system and of the user, since the former are in control of the
designer while the latter must be accepted as given. Furthermore, as described in the section
on presence theory, several authors ascribe a unique role to interaction as a key element of
presence, arguing for paying special attention to interactivity factors.
69

Chapter 4
4.4.1 VR characteristics
Vividness
According to Steuer, vividness refers to the ability of a technology to produce a sensorial
rich mediated environment. In this article this category has a broader meaning than Steuer
originally intended and also includes factors related to the content of a VE, since these can be
closely related to sensory fidelity of the display used for showing this content.
Several empirical studies have found a relationship between aspects of VE vividness and
presence. The results of these studies have been summarized in table 4.1 and are further
elaborated in the following paragraphs.
(Axelsson et al., 2000) found subjects to have a significant higher sense of presence in a
CAVE when compared to desktop VR; 44 subjects were required to solve a threedimensional
puzzle together in VR, half of them using a CAVE-system while the other half
used desktop VR. Presence was measured using a single question.
(Barfield et al., 1998) reported a significant effect for update rate in a study involving eight
subjects in a within-subject design. The six conditions were based on combinations of two
variables: type of input device and update rates of 10, 15 and 20 Hz. Presence was measured
mainly through 1 question.
(Prothero & Hoffman, 1995) found a weak but significant effect for Field Of View (FOV).
The study involved 38 subjects in a within-subject design with two conditions: 60 0 FOV and
105 0 FOV. Presence was measured using a five-item questionnaire.
(Kim & Biocca, 1997) however, did not find any effect for FOV in an experiment involving
96 subjects, exposing them to an infomercial on TV in a between-subject experiment with six
conditions based on two variables: FOV and illumination of the real environment. FOV were
chosen of 9.8 0 , 21.5 0 and 33.7 0 . Presence was measured using the Kim & Biocca
questionnaire. Illuminating the real environment, making it more distracting, also had no
significant effect on presence.
(Prothero et al., 1995a) reported a significant effect for foreground/background
manipulations: the screen was masked either near the eye or near the computer screen. In
this within-subject experiment 39 subjects participated with the two conditions of foreground
or background masking. Participants reported higher presence for the eye masking on a fiveitem
questionnaire.
(Hendrix & Barfield, 1996a) found significant effects for Geometric Field Of View 7
(GFOV) and stereoscopy in a study with a within-subject design involving 12 subjects.
Subjects participated in three consecutive experiments in which one of three variables was
manipulated: stereoscopy, head tracking and GFOV of 10 0 , 50 0 or 90 0 . Presence was
measured using a two-item questionnaire.
(Freeman et al., 2000) also found a significant correlation between stereoscopy and presence,
as well as between presence and video motion; 24 subjects in a within-subject design were
exposed to four conditions based on two variables: stereoscopy and moving or still images.
Presence was measured using an analogue rating scale. Subjects were required to mark their
level of presence with a dot. The distance between the dot and the bottom line was taken as a
measure for presence.
7
GFOV is the view-angle represented on the screen, which can be independent of the screen
size or regular FOV.
70

Presence
Presence Finding (se = Experiment design
Factor
measure significant effect) C design n
CAVE vs. Desktop (Axelsson et 1 question se: F=62.60 2 between 44
al, 2000)
subject
Update rate (Barfield et al, 1998) 1 question se 2x3 within
subject
8
Field of View (Prothero & 5 item
se 2 within 38
Hoffman, 1995)
questionnaire
subject
Field of View (Kim & Biocca, Kim & Biocca no se 2x3 between 96
1997)
questionnaire
subject
Illuminat.real env. (Kim & Biocca, Kim & Biocca no se 2x3 between 96
1997)
questionnaire
subject
Fore-/backgrnd (Prothero et al, 5 item
se 2 within 39
1995a)
questionnaire
subject
Geometric FoV (Hendrix & 2 item
se 2 within 12
Barfield, 1996a)
questionnaire
subject
Stereoscopy (Hendrix & Barfield, 2 item
se 2 within 12
1996a)
questionnaire
subject
Stereoscopy (Freeman et al, 2000) Single
se: F=17.025 2x2 within 24
analogue rating
subject
Video motion (Freeman et al, Single
se: F=29.041 2x2 within 24
2000)
analogue rating
subject
Animation (Schubert et al., 2000) IPQ no se 2x2 between
subject
56
Pictorial realism (Welch et al., 1 question se:F=35.66 2x2x2 within 20
1996)
subject
Pictorial realism (Welch et al., 1 question se:F=4.52 2x2x2 within 20
1996)
subject
Dynamic shadows (Slater et al., Slater et al. se only for 2 within 8
1995)
questionnaire + visually
subject
behavioral
measure
dominant people
Spatialized sound (Hendrix & 2 item
se 2 within 16
Barfield, 1996b)
questionnaire
subject
Tactile augment. (Hoffman et al., 1 question se 2 within 14
1996)
subject
Visual detail (Dinh et al, 1999) 14 item
no se 2x2x2x2 between 322
questionnaire
subject
Olfactory cues (Dinh et al, 1999) 14 item no se: F=2.401 2x2x2x2 between 322
questionnaire
subject
Auditory cues (Dinh et al, 1999) 14 item
se: F=36.66 2x2x2x2 between 322
questionnaire
subject
Tactile cues (Dinh et al, 1999) 14 item
se: F=18.30 2x2x2x2 between 322
questionnaire
subject
Meaning in VE (Hoffman et al, 4 item
se 2 within 33
1998)
questionnaire
subject
Table 4.1: Overview of empirical studies on vividness factors causing presence. About the
design, the number of conditions (c) is mentioned, divided into the different factors
manipulated in the study, as well as the overall design (within or between subject) and the
total number of subjects (n). The conditions manipulating the factor mentioned in the row are
marked in bold and underlined.
71

Chapter 4
(Schubert et al., 2000) found no significant effect for animation in the form of doors
opening and comic-strip-like shoes walking in and out. The between-subject study involved
56 subjects and four conditions based on two variables: animation and self-movement.
Presence was measured using the IPQ.
(Welch et al., 1996) reported a significant effect for pictorial realism. In a study with 20
subjects and 8 conditions in a within-subject design the method of paired comparison was
used, a procedure where the subjects are exposed to two VEs and are required to indicate
which generated a higher sense of presence. Combinations of 3 variables determined the
conditions: pictorial realism, interactivity and the order in which the two VEs were
presented. Presence was measured using a single question. A second almost similar study
confirmed these findings. The conditions for this experiment were determined by: pictorial
realism, feedback delay and order.
(Slater et al., 1995) reported on the effect of dynamic shadows (i.e. shadows that change in
real time when the scene changes). In a within-subject study 8 participants were exposed to
VEs with or without dynamic shadows. Presence was measured using the Slater &
colleague's questionnaire and a behavioral test as described in the section on measures. In
general, the shadows did not have a significant effect on presence, unless the person was
visually dominant (see the section on user characteristics). Note however, that this sub-group
has an even smaller population than the already small sample in the entire study, and that
including the dynamic shadows reduced the speed of the VE dramatically, possibly creating
artifacts in the measurements.
Based on reports from suddenly deafened adults, (Gilkey & Weisenberger, 1995) conclude
that "hearing may play a special role in perception, which strengthens the coupling between
the observer and the environment, and enhances the sense of presence."
(Hendrix & Barfield, 1996b) reported a significant effect of spatialized sound when
compared to no sound or even to non-spatialized sound; 16 subjects were used in two withinsubject
experiments. The first experiment compared no sound with spatialized sound, the
second compared spatialized sound with non-spatialized sound. Presence was measured
using two questions.
(Hoffman et al., 1996) reported a strong significant effect for tactile augmentation; 14
subjects participated in a within-subject design with two conditions. In one condition the
subjects could see a ball, in the second condition they could also touch it because a real ball
was placed in exactly the same position as the virtual one. Presence was measured using one
question.
(Dinh et al., 1999) performed an extensive study with 322 participants in a between-subject
design. There were 16 conditions, based on combinations of 4 variables: high or low visual
detail, olfactory cues, auditory cues and tactile cues. Presence was measured using a 14-item
questionnaire. An ANOVA showed significant main effects for auditory and tactile cues.
Olfactory cues and higher visual detail did not have a significant effect on presence. No
interactions were found between the variables, suggesting additional sensory cues work in a
simple additive fashion on one's sense of presence.
(Hoffman et al., 1998) found that when chess-pieces in a VE were positioned in a
meaningful way, this contributed to a significantly higher sense of presence for experienced
chess-players in an experiment involving 33 subjects of four categories: non-chess players,
weak players, strong players and tournament-level chess players.
72

Presence
Interactivity
Steuer defines interactivity as "the extent to which users can participate in modifying the
form and content of a mediated environment in real time." Especially the ecological theories
place a large emphasis on the role of interaction in creating presence. The results of studies
relating interactivity to presence have been summarized in table 4.2. Further details are
provided in the following paragraphs.
Presence Finding (se = Experiment design
Factor
measure significant effect) c Design n
Interaction (Welch et al., 1996) 1 question se: F=14.00 2x2x2 Within
subject
20
Feedback delay (Welch et al., 1 question se: F=30.94 2x2x2 Within 20
1996)
subject
Input device type (Barfield et al, 1 question no se 2x3 Within 8
1998)
subject
Body movement (Slater et al., Slater et al. se 2x2 Between 20
1998b)
questionnaire
subject
Head tracking (Hendrix & 2 item
se 2 Within 12
Barfield, 1996a)
questionnaire
subject
Head tracking (Schubert et al, IPQ se 2x2 Between 56
2000)
subject
Illusory interaction (Schubert et al, IPQ se only on SP 2 Between 26
2000)
subject
Walk-in-place (Slater, 1993) Slater et al. No se 2 Between 16
questionnaire
subject
Social cues (Thie & van Wijk, Questionnaires no se 2 Between 48
1998)
subject
Table 4.2: Overview of empirical studies on interaction factors causing presence. About the
design, the number of conditions (c) is mentioned, divided into the different factors
manipulated in the study, as well as the overall design (within or between subject) and the
total number of subjects (n). The conditions manipulating the factor mentioned in the row are
marked in bold and underlined.
(Welch et al., 1996) reported a significant positive effect for interaction in general and a
negative effect for feedback delay in the two experiments already described in the previous
section on vividness. Interaction was manipulated by either letting the subject drive a car or
be a passive observer. Feedback delay, the delay between a user action and the response of
the display to that action, was set at either the minimum possible with the equipment (which
was 200-220 msec) or at an additional 1.5 seconds.
(Barfield et al., 1998) did not find a significant effect for type of input device when
comparing a three Degrees-Of-Freedom (DoF) joystick with a three DoF space mouse in the
study already described in the section on vividness.
(Slater et al., 1998b)found a positive significant effect for body movement; 20 subjects were
used in a between-subject design with four conditions. The VE portrayed an area with plants,
some of which had leaves with discolored undersides. All subjects were given the task of
counting the diseased plants. For half the subjects, the plants were of similar height and
could easily be inspected while looking at eye height in standing position, for the other half
there was greater variance in the height of the plants and the subjects had to move their
73

Chapter 4
bodies in order to see the undersides of the leaves. Half of the subjects were given the extra
task of also remembering the location of the diseased trees.
(Hendrix & Barfield, 1996a) reported a highly significant effect for head tracking in an
experiment already described in can the section on vividness.
(Schubert et al., 2000) performed two studies. The first was already described in the section
on vividness and also showed a significant effect for head tracking. The second experiment,
with 26 subjects in a between subject design, had two conditions: either the participants were
told that the animations in the VE were independent of their actions or they were told the
animations responded to the user's actions (it was not said in which way). This illusory
interaction did not have a significant effect on overall presence. It did however have a small
but significant effect on Spatial Presence (see the IPQ).
(Slater, 1993) compared hand-directed locomotion to a locomotion technique called
walking-in-place, where the user can move forward by making a walking motion. 16
subjects completed this between-subject study. Presence, degree of nausea and extent of
association with the Virtual Body (VB) were measured using post-test questionnaires.
Results showed that for those using the walking in place technique, presence was correlated
with association with the virtual body presented in the VE. For those using hand-directed
travel, this was not the case.
A special case of interaction in VEs is interaction between users. A distinction is made
between presence and social presence or co-presence: the sense of being together in the
virtual world. Social presence is thought to be a part of overall presence. This is supported by
at least two studies. (Slater et al., 1998a) found a significant positive correlation between
presence and co-presence in an experiment with 30 subjects. Similarly, (Thie & van Wijk,
1998) found a significant relationship between presence and co-presence (r=0.458) in an
empirical study with 48 subjects using desktop VR. In this experiment, social cues however
were found to have no significant effect on either social presence or presence. Social cues in
one condition were limited because users could not pick their own avatar or nickname, nor
could they make gestures. In the second condition this was made possible.
However, in the study by (Axelsson et al., 2000) described in the section on vividness,
presence and social presence were not found to be related. Subjects reported higher presence
in the CAVE system, but no higher social presence.
4.4.2 User characteristics
Different individuals when being confronted with the same VE can still experience different
levels of presence because of individual differences. (Steuer, 1992) mentions the willingness
to suspend disbelief which is necessary for experiencing presence. (Witmer & Singer, 1998)
have constructed the Immersive Tendencies Questionnaire (ITQ). Cluster analysis of data
filled in by 152 subjects revealed three subscales:
• Involvement, the propensity of subjects to get involved passively in some activity, such
as reading a book.
• Focus, the ability to concentrate on enjoyable activities and block out distraction.
• Games, the frequency with which the subject plays games and the level of involvement
in these games.
Out of four experiments, only two found a significant correlation between the ITQ and the
Presence Questionnaire (PQ), also developed by (Witmer & Singer, 1998). Combining the
data across experiments showed a significant correlation between ITQ and PQ.
74

Presence
(Slater & Usoh, 1993) use the therapeutic technique known as NeuroLinguistic Programming
(NLP) to characterize the user's psychological perceptual system. The NLP model claims that
subjective experience is encoded in terms of three main representation systems: visual (V),
auditory (A) and kinesthetic (K) and that people have a tendency to prefer one system over
the others. Furthermore, when a person represents a memory they tend to choose one of three
perspectives: first, second (from another person's view) or third (from an abstract,
nonpersonal view) person. Determining someone's preferred representational system and
perspective can be done by analyzing their choice of words. A between-subject study with
17 subjects and two conditions (with or without a virtual body) was performed. A regression
model showed positive significant correlation between presence and use of the visual
representation system as measured by counting the relative number of visual predicates the
subjects used in a small description of their experience in the VE. This could be explained by
the fact that the VE was mainly visual. A higher proportion of auditory predicates, indicating
a preference for the auditory representation system, resulted in significantly lower
presence. Use of the kinesthetic representation system showed a positive significant
correlation with presence when a virtual body was used and a negative significant correlation
when no virtual body was used. The level of presence also increased with the first
perceptual position. To determine whether preference of predicates determined presence or
the other way around, a second experiment was performed with 6 subject where the subject
had to write a report before they entered the VE. The regression model based on these results
did not successfully predict presence. (Slater et al., 1994) performed another experiment with
24 subjects where the representation system and perceptual position preferences were
measured using a Likert-scale questionnaire filled in by the subjects prior to immersion in
the VE. The level of presence was positively significantly associated with V and K (a virtual
body was included in the VE), and negatively associated with A.
(Slater et al., 1995) in a later study already described in the section on vividness, found that
including dynamic shadows only increased presence when the subject had a preference for
the visual representation system as measured by an updated version of the questionnaire used
in the previous experiment.
(Bangay and Preston, 1998) measured presence at a public VR ride. Two age groups were
represented in greater number: between age 10 and 20 and between age 35 and 45. The older
age group "tends to provide lower scores for immersion and excitement consistently."
4.4.3 Conclusions on causes of presence
The empirical research on the causes of presence, although sometimes not consistent, has
provided more insight into which factors are important for creating presence. On all three
types of causes, vividness, interactivity and user characteristics, empirical evidence has been
found for the influence of several causes. Increasing the number of modalities in which the
user is addressed can increase the sense of presence. Increasing the quality of the display
with which the user is addressed in a certain modality can also increase the sense of
presence, such as increasing the FoV or the resolution of the screen or adding a spatial
dimension to stereo sound. However, there is some evidence that there is a limit to how
much quality of displays can increase presence. Allowing the user to have interaction with
the VE has also been shown to add to the sense of presence. However, which aspects of
interaction are important to generate a sense of presence is not yet clearly understood.
It is important to note that in the aforementioned research, the construct presence was often
operationalised by questionnaires that were not proven reliable or valid. Conclusions on the
75

Chapter 4
causes of presence have therefore to be taken with some precaution. For further research the
use of valid and reliable presence-questionnaires like the IPQ or ITC-SOPI, is recommended.
4.5 The role of presence in VRET
Based on our review of the research on presence, we can now have a tentative understanding
of why VRET is effective, and what factors could contribute to this effectiveness.
Figure 4.1 shows an overview of the factors contributing to the effectiveness of VRET.
Based on our TA we can conclude that, for VRET to be effective, the patient must at least
experience some level of fear. This fear results from the patient being confronted with a
virtual anxiety-provoking situation. However, it is not sufficient that the patient simply sees
and hears this virtual situation in the HMD, the patient must also experience a sense of being
VR characteristics
Patient characteristics
View of VE
Presence
in the anxiety-provoking situation. In other words: he or she must have a sense of presence.
As discussed earlier in the section on results of presence, presence and fear may be linked to
the number of therapies that fail. We must therefore provide each patient with a sufficiently
high sense of presence, which depends in part on the individual characteristics of the patient.
Several of these characteristics, such as the nature of the phobia of the patient, also directly
contribute to the fear a patient experiences.
Fear Effectiveness
Figure 4.1: Factors contributing to the effectiveness of VRET.
The sense of presence is also determined by the characteristics of the VR setup, such as the
vividness (e.g. number of modalities and quality of the displays, design of the virtual world)
and the interactivity (e.g. the number and type of input devices and types of interaction
techniques) . The VR characteristics also determine what can be seen in the VE, and as
mentioned before, this also has an effect on the patient's fear: no matter what level of
presence the patient experiences, if the situation portrayed in the VE is not something the
patient fears, then the patient will not become afraid.
If fear is really essential for the effectiveness of VRET, we must find ways of controlling and
possibly increasing it. We must however assume that the patient's characteristics are given:
altering these falls very much out of the scope of this dissertation, if it is possible at all.
Hence, to increase the effectiveness of VRET we will therefore limit ourselves to changing
the VR characteristics, and here we see two possibilities: We could increase the fear level by
creating a VE that is more frightful, or we could increase the fear level by increasing the
sense of presence, for instance by changing the interaction with the VE.
There are however some limits to increasing the frightfulness of the VE. First of all, if the
patient does not experience any sense of presence, then the patient will never experience any
fear, no matter what the content of the VE. Second, when making the VE extremely fearful,
it is inevitable that the VE will also eventually become less realistic, simply because in
reality even people without phobias would not go into such frightful situations. This could
have a negative effect on the sense of presence, and could also decrease the patient's
willingness to undergo the therapy. It could also decrease the effectiveness of the therapy
76

Presence
because the situations that the patient has habituated to are not at all similar to the situations
encountered in reality. There is also a third, more philosophical objection to creating
extremely fearful situations: if we habituate people to such situations, are we not in fact
conditioning people to become fearless in situations where fear is in reality a very healthy
response?
Thus, probably the most feasible way to increase the effectiveness of the therapy by
improving the VR characteristics is by increasing the sense of presence. Our review of the
research on the causes of presence shows that a great deal is already known about the
relationship between the vividness of a VE and the sense of presence. However, much less is
known about the effect of the interactivity of the VE, of the HCI.
4.6 HCI and the effectiveness of VRET
Therapist-Computer Interaction
Patient-Computer Interactions
View of VE
Presence
Fear Effectiveness
Figure 4.2: The relationship between HCI and the effectiveness of VRET.
In VRET, both the interaction between patient and computer and the interaction between
therapist and computer can have an effect on the fear a patient experiences and thus on the
effectiveness of the therapy. As depicted in figure 4.2, the therapist can achieve this by
changing aspects of the patient's view of the VE. This can be a change in the viewpoint of the
patient in the VE, or a change in certain elements of the VE (e.g. closing a door in a
claustrophobia application or turning on the turbulence in a fear of flying application). The
patient can also, to some extend, change what he or she sees in the VE, for instance by
changing his/her viewpoint, thus changing his/her fear. But the interaction between patient
and computer can also have an effect on presence, and this way influence the level of fear.
Both the ecological perspective and the embodied presence model previously discussed in
this chapter consider interaction to play an essential role in developing a sense of presence.
The ecological perspective bases this claim on the fact that humans perceive their
environment in terms of its affordance, the possibilities for interaction that a person has.
These perceived possible interactions are part of the user's mental model of how the
environment, and in the case of VRET, how the VR works. The embodied presence model
then further stresses that for presence to be experienced, this mental model should include
the perceived possibility to navigate and move the own body in the VE.
If interaction is to help the user in building a mental model of the VE, it is necessary for the
user to comprehend the interaction. In the words of (Zahorik & Jenison, 1998), the
interaction should be ‘lawful’: responses from the environment must be similar to those in
the real-world environment in which our perceptual system evolved. (Mantovani & Riva,
1999) further stress that the interaction should be according our cultural expectations. Thus,
whether interaction can help the user in forming a mental model depends in part on the
cultural background of the user.
77

Chapter 4
Furthermore, most theories on presence suggest a strong relationship between presence and
attention. Attention is a necessity for interaction: interacting usually requires one's attention.
If the HCI goes well, the user will be captivated by the VE where he or she must interact.
However, this does require that the interaction medium is ready to hand, that it is invisible to
the user. If the medium through which the user interacts with the VE becomes visible to the
user and requires his or her attention, than this could distract the user's attention away from
the VE. Therefore, we can assume that interaction can increase presence by focusing one's
attention on the VE, or can decrease one's presence by distracting one's attention from the
VE.
Recapitulating, we believe that interaction can influence presence through:
1. The forming of a mental model by the user of the VE, where the user is positioned in the
VE.
2. The focusing of the user's attention on the VE
4.6.1 The PGMC model
In order for interaction to influence the forming of a mental model and the focusing of the
user's attention, several criteria have to be met. We can distinguish several elements that
determine whether interaction enhances or possibly even decreases presence, and these are
depicted in figure 4.3:
Participant. Interaction will only enhance a
user's mental model if it is comprehensible to
that user, if it fits his/her previous believes and
understanding. Also, different people will find
different thing involving.
Goal. The goal for which a user has entered the
VE plays an important role in what the user will
find involving; interactions that are necessary
Goal
Participant Medium
Figure 4.3: The PGMC model.
for the user to fulfill that goal are more likely to capture one's attention.
Content
Medium. The medium through which the user interacts with the VE (i.e. the displays and
input devices) can limit the forming of a mental model and divert the user's attention away
from the VE. At best, the medium is invisible to the user: it is ready-to-hand. For this it
should fit the expectations of the user, which are in part determined by the cultural
background of the user.
Content. The content of the VE is the situation where the user must feel present in. It must
offer perceivable possibilities for interaction. Especially, based on the embodied presence
model, it must offer at least the perceivable possibility of moving and navigating one's own
body in the VE.
Together, insight into the Participant, Goal, Medium and Content (PGMC) can help us in
determining the effect interaction will have on presence.
There is some similarity between the PGMC-model and widely accepted criteria for
usability: A key concept for creating usability is to know the user, which in this case
conforms with the participant. One should design with the user’s goals in mind, conforming
to our aspect labeled goal. A highly usable interface is one that is ready to hand, conforming
to an invisible medium. Finally, a usable system should offer perceivable affordances,
similar to the content of a VE. However, a big difference is that the criteria for designing for
78

Presence
usability are focused on the user achieving his or her goals, whilst the presence model is
aimed at the experience that is provided for the user.
4.7 Conclusions
In this chapter we have reviewed previous research on presence to determine how VR can
effect the user in a non-cognitive way. We have found that there is some evidence supporting
the notion that presence is essential for people to become afraid in a VE, and that this fear is
needed to treat people in the case of VRET. However, this evidence is not conclusive. It has
not yet been shown that presence as measured using questionnaires has a causal relationship
with the fear people experience, and this fear has not been proven to be essential for the
effectiveness of VRET, even though based on our TA we must conclude that it is.
We furthermore looked at the role of HCI in the effectiveness of VRET, and have proposed
that both patient and therapist can increase or decrease the fear a patient experiences by
altering what the patient's sees of the VE, but that for the patient, his or her interaction also
can have another effect on his or her fear by changing the sense of presence the patient
experiences. However, although there is a great deal of evidence linking the vividness of a
VE to presence, there is yet little understanding of the role of interactivity in generating a
sense of presence. We have proposed that interaction effects presence by helping the user
form a mental model of the VE and by focusing the user's attention on the VE. We
furthermore proposed the PGMC model to identify those aspects that determine what effect
the interaction will have on presence.
Using the theory reviewed and developed in this chapter, we can now identify those aspects
of the HCI that might contribute to increased effectiveness of VRET.
79

5 Patient's user interface
For the patient’s UI the most important aspect is the experience of the user. Non-cognitive
factors play an important role here, linked to one of the usability problems identified in
chapter 3: how to let the patient experience a sufficient amount of fear? Other, more
traditional usability problems such as how to prevent the user from stumbling into objects he
or she cannot see with the limited field of view must also be taken into consideration. A
further restriction on a possible design of this UI and the subsequent user interaction is that
use of the system does not lead to excessive amounts of simulator sickness.
But what exactly does the interaction in VR entail? In general, the three most important types
of interaction in VR are:
• Navigation
• Object selection
• Object manipulation
In current VRET applications, object selection and manipulation do not play an important
role. To expose someone to anxiety provoking situations such as great heights or small
spaces, it is sufficient for that person to navigate to that situation in the VE. Even though
object selection and manipulation might add to a sense of presence, we will focus primarily
on navigation in VR, since this is the most common interaction type in most VEs in VRET
and other applications as well.
Navigation can be divided into (Bowman, 1999)
• Locomotion : the control of the viewpoint
• Wayfinding : the cognitive process of determining a path
In this dissertation, wayfinding techniques will not be discussed, since the virtual
environments used are relatively simple, and we assume that wayfinding has no therapeutic
value.
The method of locomotion therefore is our most important design parameter when designing
the UI for the patient. But what requirements are applicable to the locomotion method?
5.1 Requirements
These requirements should follow from our TA and from our model regarding the concept of
presence. Our TA showed that navigation in VRET had one sole purpose: to alter the fear
level of a patient. When the patient does not experience enough fear, he or she is moved to a
scarier situation. If the fear is to high, the patient is moved away to a less fearful situation.
The locomotion technique must enable the patient to experience the correct level of fear, no
more and no less. In the previous chapter we discussed two ways in which interaction could
influence fear: by changing what the patient sees or hears and by creating a sense of
presence.
A user can change what he or she sees and hears insofar as the locomotion technique allows
for this. It should provide users with means to control their viewpoint. The level of control
that a user has is dependent on the possible variables that the user can influence, such as
location and rotation, and the accuracy with which he or she can influence these variables. A
81

Chapter 5
locomotion technique suitable for VRET should therefore provide the patients with control
preferably over both location and rotation and this control should have a certain level of
accuracy.
Interaction can create a sense of presence. According to our model of the way interaction can
influence presence and fear in a VE as described in the previous chapter, the locomotion
technique is part of the medium through which the user interacts with the VE. To create a
sense of presence, the medium should be invisible to the user, or ‘ready-to-hand’. In other
words, it should be natural to use.
Based on the fact that simulator sickness is most likely based on sensory conflict, we can
imagine that different interaction techniques can result in different sensory conflicts and thus
in different levels of simulator sickness. One further constraint we must therefore impose on
the locomotion technique is that it does not add to the simulator sickness experienced by the
patient.
Beforehand, we can already assume that we will need a locomotion technique that offers a
first person perspective. According to the presence theories mentioned in the previous
chapter, people will feel present when they perceive themselves to be in a specific situation.
Seeing a virtual body in third person perspective will most likely be less involving, less
immediate and in general lead to less presence and anxiety.
Second, we will also assume that we need a locomotion technique that uses a metaphor
similar to walking since most anxiety provoking situations for phobics are in situations
where they can walk (with the exception of phobias such as the fear of driving). Other
metaphors such as driving and especially flying seem far less appropriate, since these differ
too much from the real world situation.
5.2 Design options
The first and most obvious choice we can make about the patient locomotion is whether the
patient or the therapist has control of the patient's position in the VE. In the case study
described in the task analysis, the patient could control the rotation and for a very small part,
the position of the viewpoint through use of the headtracking but needed to communicate
with the therapist to control the location further. Instead of this configuration, found in
several of the commercially available VRET system, we could also give the patient full
direct control over both location and rotation, as is the case for several other VRET systems.
This full control in the hands of the patient most likely has a positive effect on sense of
presence, since it easier for the user to change his or her position. In other words, the
medium has become more natural, more ready-to-hand. Our hypothesis is therefore
Hypothesis 1: Locomotion controlled by the patient will increase the patient’s sense of
presence.
In chapter 4 it was noted that no research to date has yet proven the hypothesis that presence
can indeed directly influence the fear a person experiences. As a control we will therefore in
this case test the consequence of assuming this hypothesis to be true:
Hypothesis 2: Locomotion controlled by the patient will increase the fear a phobic user can
experience.
82

Patient’s user interface
If this proves to be the case, then the next choice about the patient locomotion is equally
obvious: what type of locomotion technique should we provide the patient with? According
to the requirements discussed above, the locomotion technique should be one that is most
natural. That way the patient will experience the highest sense of presence and thus will be
able to reach higher levels of fear in a VE. Our hypothesis is therefore:
Hypothesis 3: A more natural locomotion technique for the patient will increase the presence
a user experiences
Again, if higher presence does indeed lead to higher levels of fear in the patient, we can also
state that:
Hypothesis 4: A more natural locomotion technique for the patient will increase the fear a
phobic user can experience.
We will need to take a look at the locomotion techniques that are available. A locomotion
technique is based on input and output devices, and the way the systems responses to the user
input.
5.2.1 Input devices
Pressure Position
Linear Rotary
X Y Z rX rY rZ
Bird
Joystick
Mouse
3
4
1 ∞ 1 ∞ 1 ∞ 1 ∞ 1 ∞ 1 ∞
Figure 5.1: Taxonomy of input devices. Adapted 8 from (Mackinlay
et al., 1990) as quoted in (Steed, 1996).
Input devices transduce a physical property into a signal that can be registered by the
computer. Figure 5.1 shows a taxonomy for input devices, including the characterization of
three specific devices often used in VR: the Flock of Birds, a regular three-button mouse and
an analog 4-button joystick. The Flock of Birds is a magnetic tracking device that consists of
8 The taxonomy has been simplified. First of all, the original taxonomy distinguished
between relative and absolute position. Because position can always be called relative this
caused different classifications for the same device. See for instance (Steed, 1996),
(Kalawsky et al., 1999) and (MacKenzie, 1995). Second, the original taxonomy
distinguished several more levels of sensitivity, which in the opinion of this author did not
add to the expressiveness of the taxonomy.
Torque Rotation
83

Chapter 5
a base transmitter that generates a magnetic field and a receiver from which the relative
position and rotation of that receiver can be derived.
Each circle in the diagram represents a transducer of a physical property. The classification
is:
• Linear or rotary
• Position or pressure for linear transducers
• Rotation or torque for rotary transducers
• Direction: X, Y or Z
• Sensitivity:1 or ∞
Sensitivity indicates the different states between which the transducer can discriminate. A ‘1’
indicates two states, usually ‘on’ and ‘off’, the symbol ‘∞’ indicates a range of states.
A solid line indicates that the device integrates several transducers into a single one. A dotted
line indicates a device that is in fact composed of several devices. For instance, a mouse is
composed of a device for measuring x and y position simultaneously and devices for
registering pressure, i.e. the mouse buttons.
For immersive VR, often magnetic tracking devices such as the Flock of Birds are used.
However, these have the disadvantage of being quite expensive and sensitive to
electromagnetic interference. Furthermore, they usually have a very limited range.
5.2.2 Output devices
There are three main VR configurations: desktop VR, HMD and CAVE. These are based on
three different types of output devices: a standard monitor, an HMD and a multi-projectionscreen
setup. Of these output devices, the HMD has been the main choice of developers of
VRET applications.
When compared to desktop VR, the HMD has the advantage of immersing the patient
completely, thus possibly generating a much higher sense of presence. Compared to a CAVE
system, HMDs have the advantage of being much cheaper and taking in much less space.
The main disadvantage of an HMD is that it usually very uncomfortable to wear.
Nevertheless, for the foreseeable future the HMD seems to be the optimal display device for
VRET.
5.2.3 Locomotion techniques
Apart from the input and output devices, the HCI related to locomotion is also determined by
the way the system responds to the input, resulting in changes in the viewpoint in the VE,
displayed in the output devices. This behavior of the system, which depends in part on the
selected devices, is the locomotion technique.
A great variety of locomotion techniques exist which translate the input of the users to a
change in the viewpoint (Mine, 1995). Figure 5.2 shows a taxonomy of locomotion
techniques.
In general, a locomotion technique is designed by choosing a method from each of the three
areas: direction / target selection, velocity / acceleration selection and input conditions.
84

Direction / target selection
Velocity / acceleration selection
Input conditions
Gaze-directed steering
Pointing / gesture Steering
Discrete selection
2-D pointing
Constant velocity and /or acceleration
Gesture-based
Explicit selection
User / environment scaling
Automatic / Adaptive
Constant travel / no input
Continuous input state
Start / stop input
Start input / stop automatically
From lists
Patient’s user interface
Environmental / Direct targets
Discrete (choose 1 of N)
Continuous (choose from range)
Time-based
Position based
Figure 5.2: Taxonomy of Virtual Locomotion Techniques (Bowman et al., 1998).
5.2.4 Locomotion in immersive VR
Locomotion techniques in immersive VR often use tracking devices instead of more
traditional input devices. The most common technique is the use of a tracker attached to the
HMD to control rotation and translation of the viewpoint with a one-to-one mapping between
magnitude of rotation and translation in the real world and the virtual world. Position
tracking has limitations concerning the distance that can be traveled, because either the
tracker itself has a limited range, the HMD is attached to a cable or the real world space in
which to move is limited. To move greater distances in the VE, often another locomotion
technique is combined with the headtracking to allow the user to move greater distances in
the VE.
Besides tracking the head of the user, more tracking devices can be used to track body
movement. A minimum of four trackers is required to create a good approximation of body
posture, tracking head, hands and torso (Badler, 1993). However, this is seldom done
because of the prohibitive costs of the extra tracking devices required.
Based on our criteria for the locomotion technique in VRET that the user should have control
over both location and rotation, and that we use a first person perspective and a walking
metaphor, we can make a preliminary selection of those techniques that match our criteria:
1. Hand-controlled viewing: Moving and turning in the VE is performed using a handcontrolled
device such as a joystick, space-mouse or trackball. This technique is most
often used for non-immersive VR, and is currently very popular for 3D games.
2. Physical movement: By tracking the position of the user’s head, moving and tuning in
the real world results in an equal translation and rotation in the VE. This technique is
already implemented in our system, but has a very limited range (approx. 1 m. in every
direction) because of limited tracker range and limited space in the real world.
85

Chapter 5
3. Gaze-directed: This technique is similar to 2. Additionally, the user has a handheld
device with one or two buttons: one moves the user forward in the VE in direction the
user is looking, an optional second button makes the user go backward.
4. Hand-directed: The user is holding a tracked device with a button which, when pressed,
makes the user move forward in the VE in the direction that the device is pointed at.
This is the most commonly implemented mode of travel in immersive VEs.
5. Torso-directed: Similar to 4, but now the device is attached to the torso of the user. This
technique should be more natural to the user since we normally move in the direction
our torso is pointing.
6. Leaning / range: By using the tracker in the HMD, the system can detect when the user
has stepped forward or is leaning forward and can move the user in this direction. In our
case, we could give the user a limited space in which to travel using physical movement.
When the user steps outside of this range, the user will start traveling in that direction in
the VE.
7. Target selection (raycast or voice): The user can select a target, either by use of
raycasting (selecting an object by using a kind of virtual laser-pointer) or by simply
calling out the name of the object. The user will then automatically be moved to that
object in the VE.
8. Airgrab (linear or go-go): The user’s hand is tracked and the user can ‘grab’ the air and
pull it towards him/herself, thus moving him/herself forward. The relation between the
distance in the real world and in the virtual world can be linear or non-linear (go-go).
9. Walking in place (Slater et al., 1993): Using the tracker in the HMD, the system can
detect walking-in-place motions by the user and moves the user forward in the lookingdirection
determined by the headtracking.
5.2.5 Evaluations
In this paragraph the experiments that have already investigated differences between travel
techniques for immersive VR are described.
(Howarth & Finch, 1999) compared headtracking to hand-controlled viewing. A withinsubject
experiment with 14 subject showed headtracking to lead to more reported nausea.
However, in the headtracking condition a rotation of the head of 120 0 resulted in a viewpoint
rotation of 360 0 . This most likely influenced the results of this study.
(Bakker, 2001) showed, in several experiments, that headtracking, providing the user with
vestibular and kinesthetic feedback, allows for much better path integration and spatial
orientation in general than movement of the viewpoint using a space mouse. Similar findings
were reported by (Pausch et al., 1997), who showed that headtracking can improve
performance in search task, especially because the user is better at determining directions
which he or she already has viewed.
(Bowman, 1999) performed a series of experiments for his PhD research:
86

Patient’s user interface
Spatial awareness
Comparing
• Slow constant velocity
• Fast constant velocity
• Slow in/ slow out
• Instant teleportation
An unknown number of subjects performed a task, where they had to learn the layout from
the VE, consisting of several colored cubes, from one location. They were then moved to
another location using one of the four velocity schemes and where required to find the cube
indicated by a color from the new location and press one of the mouse-buttons according to
the letter on that cube.
The time to find the object was significantly longer for the teleportation technique.
Travel technique for absolute motion
Comparing
• gaze-directed
• hand-directed
The task was to move from a starting position into a sphere. An unknown number of subjects
completed this experiment. Results showed no significant difference in completion time and
accuracy.
Travel technique for relative motion
Comparing
• gaze-directed
• hand-directed
The task was to move from a starting position to a position relative to a sphere, indicated by
an arrow attached to the sphere. Results showed a significant difference in completion time.
Subjects using the gaze-directed technique had to turn their head to the sphere to check their
location, and then had to turn their head in the direction they wanted to move, resulting in
slower completion times.
Information gathering
Comparing
• gaze-directed
• hand directed
• torso-directed
and
• 1 dimensional maze
• 2 dimensional maze
• 3 dimensional maze
and
• Unconstrained navigation
• Collision detection
26 subjects participated in this within-subject experiment with 16 conditions (torso-directed
navigation is not possible in three dimensions). The task was to learn the location and
87

Chapter 5
content of labels distributed in the VE. Subjects received a score based on the completion
time and accuracy of recall of the content and position of the label. Results showed
dimensionality to have a significant effect on the score. Travel technique and collision
detection did not have a significant effect.
Travel technique comparison
Comparing
• Hand-directed
• Gaze-directed
• Torso-directed
• Target selection (raycast)
• Target selection (icon on a 2d map)
• HOMER: Selection using raycast, hand movements for velocity
• Air-grab (go-go)
A total of 38 subjects completed the experiment. Travel technique was a between-subject
variable. The VEs contained four flags with different numbers. Each subject went through
eight trails: four trials to find one of the flags while the other flags were invisible and then
four trials to find the flags again (primed search). During the last four trials, two variables
were manipulated: visibility of the flag from the starting position and the accuracy needed
for task completion.
The time between the onset of the stimulus and the beginning of the movement and the time
actually spent moving was measured, as well as the total completion time. Gaze-directed and
hand-directed steering proved significantly better on all these criteria, and target-selection
using the 2D map performed significantly worse.
5.2.6 Conclusions
According to our criteria, the locomotion technique best suited for VRET is the one that
provides the patient with the highest sense of presence and the highest levels of fear in a
given situation. The first choice we can make is whether we want to provide the patient with
locomotion control, or the therapist.
If the choice is made for patient control, then there is still a wide variety of locomotion
techniques that meet our criteria for moving the patient in VRET. Fortunately, a great
number of these have already been evaluated by other researchers in terms of the resulting
spatial awareness of the user and the speed and precision with which users can complete
tasks in the VE. Few evaluations however have taken the sense of presence of the user into
account and none have looked at the effect of the locomotion technique on fear.
5.3 Experiment 1: Patient vs. therapist control
In the first experiment patient control was compared to therapist control in terms of the sense
of presence that was induced in the subjects. In order to better understand the relationship
between interaction and presence, this experiment was also used to investigate our
preliminary conclusion based on the presence research that interaction can enhance presence
though the focusing of the user's attention and by helping the user form a mental model of
the VE. For this, measures for the subject’s attention and mental model were used.
There are in fact two differences between patient and therapist control. In the case of
therapist control, the patient not only loses the ability to move him or herself autonomously,
88

Patient’s user interface
he or she also becomes dependent on the therapist, another human being, for locomotion.
The patient will need to communicate with the therapist about where he or she needs to go.
To understand the effect of autonomous navigation on the one hand and this therapist-patient
communication on the other hand, a third condition was added to the experiment besides the
therapist control condition and the patient control condition. In this third condition, the
patient was moved through the VE automatically and therefore had no autonomous control
and also no need to communicate with the therapist.
5.3.1 Method
Design
In this experiment, three conditions were tested in a between-subject design:
• LEAN. The patient can navigate through the VE him or herself. The interaction
technique used is 'leaning': the patient is standing inside a railing of one-by-one meter.
When the patient steps or leans away from the center of this area in a certain direction,
he or she will start moving in that direction in the VE at a constant velocity.
• THERAPIST. Similar to our previously used setup for VRET, the therapist must navigate
the patient through the VE.
• PATH. The patient is navigated through the VE using an automated script. This
condition is intended to distinguish between the effect of autonomous navigation by the
patient and the effect of the patient-therapist communication.
To simplify the experiment, the task subjects had to perform was not exposure therapy.
Instead, subjects had to locate a small but clearly visible object in the VE and move close to
it. When the subject came close to the object, it would disappear and re-appear somewhere
else. The experiment was completed when the object had re-appeared a fixed number of
times. This task is thought to be similar to VRET in the sense that in VRET patients have to
find situations that are frightful and move into these situations. In the THERAPIST
condition, subjects were told that the 'therapist' could not see the objects, and therefore the
subjects and 'therapist' were forced to communicate in order for the subject to complete the
task. Again, this task is intended to be similar to current VRET practice, where the therapist
has to navigate the patient, whilst only the patient can perceive the situations that are
frightful to him/her. The therapist in this experiment was the experiment leader, who made
sure the communication was only about the VE.
Virtual Environment
The VE consisted of a 'plus'-shaped room with
several types of furniture in it. One segment of
the room was designed to look like a kitchen to
give the user a recognizable reference point.
The railing surrounding the subjects in reality
was also shown in the virtual world as visible in
figure 5.3.
The VE was generated using our VR testbed
configuration described in chapter 3.
Figure 5.3: VE used for the experiment,
with a virtual representation of the railing
surrounding the subject in reality.
89

Chapter 5
Measures
Prior to the experiment some information regarding the subjects was gathered, such as age,
gender and experience with 3D games and computers in general.
Figure 5.4: 2D representation of the VE, where subjects were
required to position markers to indicate where they thought a
certain piece of furniture was positioned in the VE.
During the experiment, a loud sound was played at random intervals. The subjects were
instructed to respond to this sound as fast as possible by pressing a button on a device they
were holding. The response time was recorded and used as a measure for the amount of
attention the subject was devoting to the VE, where a slower response time indicates the user
was more involved in the VE than when a faster response time was recorded. As a control,
the average time between the beeps was also measured.
Afterwards, subjects were required to position certain recognizable pieces of furniture on a
2D representation of the VE as depicted in figure 5.4. The accuracy 9 with which they could
recall these locations was used as a measure for the quality of the mental model the subject
had created during the exposure to the VE.
Presence was measured using the Igroup Presence Questionnaire (IPQ) (Schubert et al.,
1999). The IPQ is divided into three subscales: Involvement, Spatial Presence and Realness,
and is discussed in more detail in chapter 4.
Subjects
In all, 45 subjects participated in the experiment, four of whom were female. Most were
students. All subjects received a Fl.25 reward (approx. €11). One subject could not complete
the experiment due to malfunctions in the VR software and was excluded from our analysis.
Subjects were randomly assigned to one of the three conditions.
9 The accuracy was measures by determining the difference between the actual location and
the location indicated by the subject. A score of three points was given if this distance was
smaller than 25 pixels, two points for 25 to 50 pixels, one point for 50 to 75 pixels and zero
points for over 75 pixels of distance. The 2D map was 500x500 pixels, corresponding with a
VE of 7.5 x 7.5 meters.
90

5.3.2 Results
Patient’s user interface
Reliability
The average time between beeps was found to differ between individuals due to the random
nature of the intervals and, more seriously, was found to correlate significantly with
presence. It was therefore necessary to correct for this and include average time between
beeps as a covariate.
Cronbach’s Alpha for the IPQ was .76 (n=44), showing that responses showed internal
consistency.
Effect of condition
Average time between beeps was
used as a covariant in an ANalysis of
COVAriance (ANCOVA) with
presence as dependent variable and
condition and average time between
beeps as independent variables. This
ANCOVA did not show a significant
difference between conditions in the
scores on the presence questionnaire
(F=1.053, p =.360). Figure 5.5 shows
a boxplot of presence in each of the
three conditions.
Interestingly, within (and only in) the
THERAPIST condition, there was a
negative correlation between
presence and the duration of the
experiment (Pearson correlation = -
.574, p=.025).
People scored significantly higher on
the mental model test in the
THERAPIST condition when
compared to the other two conditions.
A post-hoc Tukey HSD analysis
showed a mean difference of 11.02
(p=.001) and 10.00 (p=.003) between
THERAPIST and PATH and
between THERAPIST and LEAN
respect-ively. Figure 5.6 shows a
boxplot of the scores on the mental
model test in each of the three
conditions.
The attention score does not differ
significantly between conditions.
This can also clearly be seen in figure
5.7.
Presence
40
30
20
10
0
-10
-20
-30
Therapist
Path
12
Lean
Figure 5.5: Boxplot of the presence scores in the
three conditions.
Mental model score
40
30
20
10
0
-10
Therapist
Path
Lean
Figure 5.6: Boxplot of the mental model scores in
the three conditions.
91

Chapter 5
Unstandardized Coefficients
Standardized
Coefficient
B Std. Error Beta t Sig.
(Constant) -5.320 9.104 -.584 .562
Reaction time 1.387E-02 .010 .219 1.386 .174
Mental model score 7.451E-02 .175 .067 .427 .672
Table 5.1: Coefficients in a linear regression model. Dependent variable: presence (IPQ).
Predictors for presence
Table 5.1 shows the coefficients for a
linear regression model with presence
as dependent variable. Neither the
mental model score nor the reaction
time are reliable predictors for the
overall presence score. Interestingly
however, there is a significant
positive correlation between reaction
time and the Involvement subscale of
the IPQ (Pearson's correlation = .321,
p=.041).
5.3.3 Discussion
We can discuss the results of the
experiment in terms of the PGMC
model.
Average reaction time (milliseconds)
1400
1200
1000
800
600
400
Therapist
Path
33
Lean
Figure 5.7: Boxplot of the average reaction times in
each of the three conditions.
Participant
The participants in this experiment were mostly students from our technical university. As
such, they in general had much experience with computers and video games. It is highly
likely that this influenced their easy comprehension of the interaction techniques and thus the
quick forming of a mental model.
Goal
The goal was the same for all the subjects: finding the small object and move close to it. For
this, two types of interaction were available: looking around by means of the headtracking
and, in the THERAPIST and LEAN condition, navigation. The goal ensured that subjects
used these two interaction types extensively, and in a way similar to a real VR exposure
therapy situation.
Other types of interaction, such as for instance manipulating objects or turning on music
would be far less interesting to subjects with the goal they were given. However, one must
always keep in mind that subjects have hidden goals, such as satisfying their curiosity about
VR or simply earning the reward paid at the end of the experiment.
92

Patient’s user interface
Medium
The factor manipulated in this experiment was related to the medium through which subjects
could interact with the VE. In the LEAN condition, the medium was a direct one where
actions by the subject would result in immediate responses from the system. In the
THERAPIST condition, subjects had to interact with the VE indirectly through
communication with the experiment leader. In the PATH condition, the medium did not
allow for any interaction in terms of navigation.
The lack of difference in presence reported by the subjects between the conditions suggest
that the differences in the medium were too small. Subjects in all conditions could still look
around through use of the headtracking, providing users with enough interaction to keep
them involved and build a mental model.
The negative correlation between presence and the duration of the experiment in the
THERAPIST condition could be explained by assuming that a short duration of the
experiment in this condition was due to efficient cooperation between the subject and the
experiment leader. In case of efficient cooperation, the medium (communication with the
therapist) would demand less attention and become more invisible to the subject, thus
increasing the sense of presence.
Interestingly, the communication with the therapist did result in a higher score on the mental
model test. This score was, however, not correlated with the presence score. One of the
scholars responsible for the development of the embodied presence framework and the
presence questionnaire suggested that this could be due to the fact that the mental model
needed for presence must be egocentric, based on possible interaction, whilst our test
assumes a more exocentric model, based on 2D layout (Thomas Schubert, personal
communication).
The attention test did not show any significant difference between conditions, again
indicating that the differences in the medium between conditions were insufficient. The
attention test did correlate with the subscale of the presence questionnaire labeled
‘Involvement’, which could suggest that the attention measured through our test plays a role
in the presence people experienced.
Content
The content of the VE supported the interaction in a straightforward manner: it provided a
space with obstacles through which the subject had to navigate and a relocating object to
serve as a target to which the user had to navigate.
The correlation between ‘average time between beeps’ and presence suggests that content
outside of the VE, in this case the beeps needed for the attention test, can reduce one’s
presence in the VE.
In general, this experiment showed that, in the limited setting of this experiment, there is no
difference in terms of presence between patient and therapist locomotion.
5.4 Experiment 2: Patient vs. therapist control
The second experiment comparing patient control and therapist control took place in a setting
more alike to real VRET practice. In a large experiment in cooperation with both the
University of Amsterdam and the VALK foundation, people were treated for fear of flying in
two VR conditions as well as other conditions not relevant to this thesis. Fear of flying is a
93

Chapter 5
phobia that often involves fear of heights or claustrophobia
(van Gerwen et al., 1997). In traditional
therapy, patients suffering from fear of flying are often
first treated for one of these two phobias if these are
found to underlie the patient’s fear of flying. Hence, in
VR, people were treated not only for fear of flying but
usually also for one of the other two phobias, giving us
the opportunity to test the two conditions in a very
diverse setting.
5.4.1 Method
Design
Two conditions were tested: therapist control versus
patient control. The therapist could control the location
of the patient by either using a path control or the
joystick. The path control allowed the therapist to
move the patient backwards or forwards on a
predetermined path. In the patient control condition,
the patient could use the lean technique also used in the
previous experiment: when the patient stepped out of
the center of the railing, he or she would start to move
in that direction at a constant velocity, either until the
patient bumped into an object or until he or she stepped
back to the center. In the patient control condition, the
therapist had no control over the patient’s location at
all.
Virtual worlds
In this experiment, a total number of nine virtual
worlds were used, which are displayed in figures 5.8
through 5.16. All subjects were exposed first of all to a
neutral virtual world that was designed to envoke no
fear in the subjects. In this neutral world, subjects
could familiarize themselves with the operation of the
system. If an analysis of a subject’s fear of flying
showed that the subject had a fear of heights or of
closed spaces, he or she was first treated for one of
these two phobias in the appropriate virtual worlds.
Otherwise, patient were immediately treated in the fear
of flying worlds.
In the airplane world, subjects were seated and did not
move through the VE other than by headtracking. In
this world there was no difference between patient and
therapist control and this VE was therefore further
ignored in the investigation described in this chapter.
94
Figure 5.8: the courtyard, a neutral
virtual world.
Figure 5.9: virtual firestairs for the
treatment of acrophobia.
Figure 5.10: acrophobia world:
rooftop terrace.
Figure 5.11: construction site for
the treatment of acrophobia.

Measures
The first world that subjects were exposed to was the
neutral world. Since no treatment took place using this
world, it was possible to use the think-aloud protocol,
encouraging subjects to verbalize what they were
thinking when navigating through the VE. These
sessions were recorded on video for later analysis to
determine any usability problems the subjects might be
experiencing when working with the system in the
specific condition.
To make a more quantitative measure of the system
usability in the different conditions, subjects were
required to fill in a usability questionnaire after the
second treatment session. This questionnaire consisted
of 14 seven-point Likert-scale items, and can be found
in appendix A. Subjects were also required to fill in a
five item questionnaire regarding their Computer
Experience (CE).
After each exposure to a virtual world, subjects filled
in the Igroup Presence Questionnaire (IPQ). Also,
before and after each exposure to a phobia world,
subject were asked to fill in the Simulator Sickness
Questionnaire (SSQ) designed by (Kennedy et al.,
1993), aimed at measuring the simulator sickness
experienced by the subject. The pre and post scores
were subtracted from each other to determine any
increase or decrease in sickness.
The subject’s fear was measured by the Subjective
Units of Discomfort (SUDs) entered by therapist,
which were recorded at a maximum interval of three
minutes, and by recording the subject’s heartrate. The
choice of heartrate as secondary measure was based on
the fact that heartrate measurement nowadays is
relatively cheap and easy to perform.
Also, during therapy the actions of the therapist were
automatically recorded, as well as the progress of the
patient through the virtual world. Most virtual worlds
were designed to slowly increase the fear level as one
moved further in the VE. Whether the patient was at
the beginning or at the end of the virtual world
therefore could have a strong relationship with the
patient’s fear.
Subjects
Persons that had enrolled for treatment of fear of flying
at the VALK foundation were asked to participate in
this study prior to their standard treatment. These
Patient’s user interface
Figure 5.12: virtual hallway for
the treatment of claustrophobia.
Subsequent corridors were
narrower and the ceilings were
lower. The therapist could dim the
lights.
Figure 5.13: Virtual closet for
treatment of claustrophobia. The
therapist could close the closet
doors, make the room and the
closet smaller and could dim the
lights in the closet.
Figure 5.14: elevators for the
treatment of claustrophobia. Therapists
could operate the elevator,
trigger an audio-visual alarm and
dim the lights in the elevators
95

Chapter 5
subjects were all diagnosed as having fear of flying.
according to DSM-IV criteria (APA, 1994). Originally,
the number of subjects to participate in the two VR
conditions of this experiment was planned at a total of
30. Unfortunately, as a result of the terrorist attacks on
the 11th of September 2001, the number of people
seeking treatment for fear of flying dropped drastically.
After almost a year, only 16 subjects had started the
experiment. These were randomly assigned to either
condition 1: therapist control (n=7) or condition 2:
patient control (n=9).
5.4.2 Results
Reliability
A reliability analysis for the various questionnaires was
performed, and the results are displayed in table 5.2.
Heartrate did not correlate significantly with SUDs
(Pearson correlation=-.022, p=.804, n=133), indicating
a discrepancy between physiological arousal and
subjectively reported fear.
Measure N of cases N of items Alpha
IPQ neutral world 15 14 .5570
IPQ session 2 14 14 .4354
CE 15 5 .6574
USA 16 14 .5479
SSQ(pre) 15 28 .8696
SSQ(post) 15 28 .8609
Table 5.2: Chronbach’s alpha as calculated for the
various questionnaires
Figure 5.15: virtual airport for the
treatment of fear of flying.
Therapists could trigger announcements
and airplane sounds.
Figure 5.16: virtual airplane for
the treatment of fear of flying.
Therapists could switch between
taxiing, takeoff, flight and landing,
could trigger messages from the
purser and the captain, switch the
fasten seatbelt sign, alter the
weather and trigger turbulence.
Presence
After each session subjects were required to fill in the IPQ. However, not all subjects had
already completed all sessions, so we will restrict our analysis to the first two sessions.
In the first session, subjects were all exposed to the same VE: the neutral world. An
ANOVA of the IPQ scores with condition as independent factor shows no significant
difference in IPQ scores between the two conditions: F=1.583, p=.230 (n=15).
During the second session, subjects were exposed to different VEs depending on the analysis
of their fear of flying. An ANOVA of the IPQ scores reported after this session shows even
less difference between the two conditions: F=.001, p=.982 (n=14).
Heartrate
In all, the heartrate of seven different subjects during 16 sessions was recorded, resulting in
3444 measures. All heartrates were corrected against a baseline, which was defined as the
average heartrate during the first 60 seconds of recording. After removing outliers at more
96

Patient’s user interface
than two standard deviation from the mean, 3314 measures remained (n=1332 in condition 1,
n=1982 in condition 2).
A linear regression showed that both the time spend in the VE and the progress of the patient
in the VE have a highly significant effect on the heartrate, as displayed in table 5.3. During
exposure, heartrate reduces over time but increases as the subject progresses further into the
VE.
Unstandardized Coefficients
Standardized
Coefficient
B Std. Error Beta t Sig.
(Constant) 19.567 .930 21.040

Chapter 5
VE does show a significant positive effect, and
was therefore included as a covariate in the
general linear model for an ANCOVA with
SUD as dependent variable and virtual world
and condition as independent factors. The
results of this ANCOVA are displayed in table
5.6, showing significant effects for both factors,
as well as an interaction between virtual world
and condition. Figure 5.18 shows a plot of the
estimated marginal means based on this
ANCOVA for the SUDs in the two conditions in
the different virtual worlds. This plot shows
i df F Sig.
(Intercept) 1 1.236 .268
Progress 1 31.638

Estimated marginal means of SUD
6
5
4
3
2
1
0
Therapist control
World
Airport
Patient control
Patient’s user interface
Firestairs
Rooftop
Construction site
Hallway
patient control condition (F=3.795, p=.073 and F=4.617, p=.051 for item 10 and item 12
respectively).
In all, 16 subjects were recorded when going through the neutral virtual world while
verbalizing their thoughts according to the think-aloud protocol. These session lasted
approximately five minutes each.
Of the seven subjects assigned to the therapist control condition, three subjects needed
explicit instructions from the therapist to turn his or her head in the direction that the subject
was moving in the VE. Two of these subjects did not move their heads at all until instructed
by the therapist, keeping their head still as if watching a monitor.
Of the nine subjects in the patient control condition, three subjects reported general
difficulties in understanding the locomotion technique. Two of these also experienced
problems making turns, where they had difficulties keeping themselves from bumping into
walls. One subject reported problems with remembering how to stop moving.
All other subjects reported no problems and usually simply commented or described the
virtual world.
Closet
Elevator
Figure 5.18: Estimated marginal means of the SUD scores for the
different virtual worlds in the different conditions.
Simulator sickness
The SSQ showed that simulator sickness was somewhat lower in the patient control
condition, but this effect was not statistically significant: F=3.069, p=.103 (n=15).
5.4.3 Discussion
The results confirm the findings of the previous experiment that presence is not higher in a
patient control condition when compared to therapist control. But even though the two fear
measures heartrate and SUDs are not correlated with one another, both do show the same
difference between the two conditions: fear is lower in the patient control condition. One
99

Chapter 5
possible explanation could be that in the patient control conditions subjects could better
avoid fearful situations. In this experiment the progress of the subjects through the VE was
taken into consideration, thus failure of the subject to enter a general area that was more
fearful did not influence the results. However, on a smaller scale subjects could avoid
moving close to a ledge or move faster through a scary part of the VE. Due to the diversity
and complexity of the VEs used in this experiment, it was not possible to take this type of
avoidance behavior into consideration.
The general usability of the system does not seem to be very different between the two
conditions. The errors reported by subjects using the think-aloud protocol indicated that the
errors patients make in the patient control condition are somewhat more serious, where
subjects have difficulties in actually achieving tasks such as making turns, compared to the
problems reported in the therapist control condition where a small number of subjects needed
to be specifically instructed to look in the direction they were being moved.
No difference in simulator sickness was found between the two conditions.
5.5 Experiment 3: Patient Control
In case the patient is given control over his or her own position in the VE, we need to
determine which locomotion is best suited for VRET. Given the large number of locomotion
techniques available to us it is infeasible to test all of them. Fortunately, especially the
previous research by Bowman has already investigated the accuracy and speed with which
users can move through the virtual world with most of the techniques. However, it is still not
very clear if a more natural technique will lead to higher presence and a higher fear. To
investigate this, we can compare the least natural technique to the most natural one.
Although of course chosen subjectively, we would like to argue that hand-controlled viewing
is the least natural interaction method, because here users have to move a device to rotate the
viewpoint and press a button to move forward, which is not at all similar to the way we move
in the real world. The most natural locomotion technique today that allows the user to move
larger distances in the VE than headtracking alone is walk-in-place. Here the user can rotate
the viewpoint by rotating his or her head and can move forward by making a walking
motion. An advantage of this technique is that is doesn’t require an extra tracking device and
is therefore relatively inexpensive.
To determine the separate effects of the headtracking and the walking motion detection, we
can add a third locomotion technique: gaze directed steering, where rotation is controlled by
headtracking and the user can move forward by pressing a button.
5.5.1 Method
Design
The experiment consisted of three conditions:
• Walk-in-place
• Trackball (hand controlled viewing using a handheld trackball device)
• Headtracking (gaze-directed steering)
The walk-in-place technique was implemented by feeding the coordinates obtained from the
tracking device into a multi-layered perceptron (Herz et al., 1993) that was trained to
distinguish between walking and non-walking behavior.
100

Patient’s user interface
Subjects were required to complete a course through a VE. Each subject was exposed to only
one condition.
Virtual Environment
The VE used in this experiment consisted of three parts
that were connected, forming a single VE: a small
training space, where subjects could familiarize
themselves with the interaction technique, a room
designed to determine the controllability of the
interaction technique and a part containing height
situations aimed at determining the effect of the
locomotion technique on the fear of the subjects.
The room designed for determining the controllability
of the technique contained several objects such as
couches and plants. A specific location in the room was
marked with a flag and a spot on the ground as
depicted in figure 5.19. The subject was instructed to
move to this marker and press a button to indicate that
he or she had reached it. After this the marker would
disappear and another would appear somewhere else in
the room. All the markers were easy to find, but
differed in difficulty to reach. This way we prevented
the subjects from losing their orientation in the VE.
Subjects were also instructed to try to avoid collisions
with the objects in the room. By instructing the patient
to avoid collision instead of telling him or her to
complete the task as quick as possible, the emphasis of
the test was put on controllability instead of speed.
When the subject had reached all seven markers, he or
she was instructed to move into an elevator, which led
to the final part of the VE. The subject was told that the
accuracy test was over and was now instructed to
Figure 5.19: First part of the VE,
with a flag as target for the subject
to move to and obstacles such as
plants and couches.
Figure 5.20: Firestair with a box
placed in the far corner, aimed at
stimulating the subjects to look in
certain directions.
locate boxes in the VE. These boxes were placed in the last part of the VE to stimulate the
subject to look around. The top side of these boxes was open and on the inside a figure was
depicted. Subjects were instructed to tell the experiment leader which figures they saw.
Figure 5.20 shows the firestairs found at the beginning of this part, along with a box.
The top section of the VE was specifically designed to measure the avoidance behavior of
the subjects. Figure 5.21 shows an overview of this section. In the first half of the section
(Situation 1), subjects could avoid the great depth by keeping to the right. In the second half
(Situation 2), an obstacle prevented them from doing this. Situation 1 and 2 are shown in
figures 22 and 23 respectively.
To avoid any unwanted effects caused by differences in framerate between different parts of
the VE and between conditions, the framerate was fixed at 15 frames per second.
101

Chapter 5
Large
drop
Large
drop
A2
Walls
obstacle
A1
Subject’s path
Situation 2
Situation 1
Figure 5.21: Overview of the top section of the
VE, showing a typical subject’s path and the
extent of the avoidance behaviour in situation 1
(A1) and in situation 2 (A2).
Figure 5.22: Situation 1, where
subjects could avoid the depth by
keeping to the right.
Figure 5.23: Situation 2, where
avoidance was hindered by
introducing an obstacle.
Measures
Prior to the experiment itself, the subjects were required to fill in the following
questionnaires:
Acrophobia Questionnaire (AQ): This questionnaire, developed by (Cohen, 1977), consists
of 40 items and can be divided into a fear and an avoidance subscale.
Motion Sickness Tendency questionnaire (MST) 10 : A short questionnaire to measure the
subject’s tendency to suffer from motion sickness.
Computer Experience questionnaire (CE) 9 : 5-item questionnaire regarding the subject’s
experience with computers and 3D programs
Tellegen Absorption Scale (TAS): Developed by (Tellegen & Atkinson, 1974), this
questionnaire aims to measure the subject’s openness to absorbing and self-altering
experiences.
Simulator Sickness Questionnaire (SSQ): This questionnaire, designed by (Kennedy et al.,
1993), is aimed at measuring the simulator sickness experienced by the subject. The
questionnaire was filled in just prior to the experiment to acquire a base-rate.
10 See appendix A for the complete questionnaires
102

Patient’s user interface
During the first part of the experiment, where the controllability of the interaction technique
was tested, the following data was automatically recorded:
• Number of collisions: The number of times the subject collided with objects in the
virtual world, and the magnitude of the collision, where a head-on collision resulted in
the highest score.
• Accuracy: The accuracy with which subjects positioned themselves near the flags in the
first part of the experiment, measured in cm.
In the second part of the experiment aimed at determining the control of fear the interaction
technique provided, the following measures were recorded:
• Heartrate: With a five second interval, the heartrate of the subjects was recorded with
an external device.
• SUDs: At certain predefined locations in the VE, the subjects were requested to rate
their fear on a scale of 0 to 10. At the beginning of the second part of the experiment, a
base-rate measure was taken to compensate for any general anxiety that the subjects
might be experiencing.
• Head-Down Rotation (HDR): Average number of degrees of downward rotation of the
subject’s head, used as an indication of whether subjects were looking down into the
depth in height situations.
• Avoidance (AV): Extend to which the subject avoided the edge of the large drop in
Situation 1 and Situation two, measured as the integral of the distance in cm from the
edge.
After the experiment the subjects were instructed to fill in the following questionnaires:
• Simulator Sickness Questionnaire (SSQ): Similar to the base-rate measured just prior to
the experiment. The final score of the SSQ is calculated by subtracting the base-rate
score from this score.
• Igroup Presence Questionnaire (IPQ): The questionnaire developed by (Schubert et al.,
1999) to measure the subject’s sense of presence.
• Usability Questionnaire (USA) 9 : Small questionnaire aimed at measuring the subject’s
subjective evaluation of the interaction technique. (see appendix A)
Subjects
To acquire a wide variety of subjects in terms of age, gender and computer experience,
representative of the typical user population for VRET, subjects were recruited through two
radio programs on Radio West, advertisement in local newspapers and in the news bulletin of
the Fobievrienden Nederland foundation, posters on the campus of Delft University of
Technology and email newsflashes at the Hogeschool voor de Kunsten Utrecht. Subjects
were excluded from the experiment if they were suffering from epilepsy, had eyesight with
deficiency greater than ± 2 dioptres (and did not wear contact lenses) , or were wearing a
pacemaker.
42 subjects qualified for these criteria, 23 women and 19 men. These subjects were randomly
distributed over the three conditions and received €10 upon completion of the experiment.
103

Chapter 5
5.5.2 Results
Subjects
Min. Max. Mean (SD) Max. Score*
AQ-fear 15 77 44.85 (16.07) 120
Computer Experience 8 20 13.66 (2.67) 25
Age 18 62 30.41 (12.42)
* Max. score = highest possible score on the questionnaire.
Table 5.7: Descriptive statistics of the subjects
Of the 42 subjects, three were not able to complete the experiment due to extreme fear,
resulting in some missing data in the fear section of the experiment. One of these subjects
was removed from the experiment altogether because of too extreme values on several
scores. Table 5.7 shows the some descriptive statistics of the remaining 41 subjects.
To test the distribution across the groups an ANOVA with condition as dependent variable
was performed for AQ (F=2.309, p=.113). Age (F=1.351, p=.271), Gender (F=.233, p=.801),
CE (F=.188, p=.830), MSQ (F=1.439, p=.250) and TAS (F=1.808, p=.178), indicating that
the subjects were evenly distributed over the three conditions.
Measure reliability
A reliability analysis has been performed
for the various questionnaires, using
Cronbach’s alpha as shown in table 5.8.
The heartrate measure was extremely
hampered by technical problems such as
magnetic interference from the tracking
device and was unusable.
The SUD scores were found to correlate
significantly with the AQ-fear scores
(Pearson correlation=.437, p=.005),
confirming that people with a higher
fear of heights report higher fear in
the VE, suggesting that the SUD
measures were reliable.
Accuracy of the locomotion
technique
Two measures are related to the
accuracy with which subjects can
move through the virtual world: the
accuracy with which subjects
positioned themselves on the markers
during the accuracy test and the
number of collisions and the
magnitude of these collisions with
objects in the room. We investigated
104
Measure N of cases N of items Alpha
IPQ 41 14 .8308
AQ 64 32 .9203
MSQ 42 5 .8380
TAS 42 6 .6632
USA 41 11 .6408
SSQ(pre) 42 28 .8269
SSQ(post) 42 28 .8572
CE 64 5 .6148
Table 5.8: Chronbach’s alpha as calculated for
the various questionnaires
Number of collisions
1200
1000
800
600
400
200
0
Walk-in-place
Trackball
13
Headtracking
Figure 5.24: Boxplot of the number of collisions in
the different conditions.

Patient’s user interface
whether age, gender or computer experience had any influence on these two measures. No
correlation was found between these variables.
An ANOVA showed that there was no significant difference in positioning accuracy between
conditions (F=2.087, p=.138), but that there was a significant difference in the number of
collisions (F=3.273, p = .049). In figure 5.24, the boxplot shows that particularly in the walkin-place
condition some people tended to have much higher numbers of collisions.
Presence
An investigation into what factors
100
could have influenced the presence
29
score revealed that there was no
90
correlation between the IPQ scores
and the AQ, gender, computer
80
experience and TAS. This is
70
interesting, since we included the
TAS because we expected absorption
60
to be important for presence. Since
50
this did not appear to be the case, the
TAS was ignored in the rest of this
40
35
study.
30
We did however find a significant
20
positive correlation between age and
Walk-in-place Trackball Headtracking
presence (Pearson correlation =.380,
p=.015), indicating that older people Figure 5.25: Boxplot of the IPQ scores.
tend to score higher on the IPQ. To
compensate for this effect in our analysis, age was used as a covariate in an ANCOVA with
condition as independent factor and IPQ as dependent factor. Our analysis showed a
significant difference in presence between conditions (F=4.722, p =.015). Figure 5.25 shows
the boxplot of the IPQ scores, showing that presence is highest in the walk-in-place
condition.
Fear
As stated previously, fear as
measured using the SUD scores
correlated with the AQ-fear scores. It
was therefore used as a covariate in
an ANCOVA with condition as
independent factor and SUD as
dependent variable. For this we used
the SUD measure prior to situation 1
and situation 2 (minus the base-rate
measure). Other variables were not
found to correlate with fear and were
therefore not included in this
analysis. Results showed a significant
difference between conditions
(F=4.015, p=.026). The boxplot of
IPQ
SUD
8
6
4
2
0
Walk-in-place
Trackball
Headtracking
Figure 5.26: Boxplot of the SUD scores.
105

Chapter 5
the SUD scores in the different conditions in figure 5.26 shows that the fear was highest in
the walk-in-place condition.
Furthermore, fear has been found to be significantly higher in situation 2 when compared to
situation 1 (F=46.402, p

Patient’s user interface
General usability
The scores on the usability questionnaire did not correlate with the scores on computer
experience, AQ, MSQ, age or gender. There also was no difference between conditions
(F=.705. p=0.500), indicating that subjects rated all locomotion techniques equally.
5.5.3 Discussion
The accuracy with which users could move through the VE did not differ greatly. Only the
number of collisions was significantly higher in the walk-in-place condition.
As discussed in the previous chapter, the locomotion technique can influence fear in two
ways: by changing what the user sees and by changing the user’s sense of presence. The
results of this experiment show that subjects indeed could use the locomotion technique to
lower their fear, resulting in lower fear scores in situation 1 when compared to situation 2.
Results also showed that all three locomotion techniques allowed for the same avoidance
behavior, since there was no difference in avoidance as measured through the distance from
the edge and the downward head rotation between conditions. This provided us with an
excellent opportunity to test whether presence does indeed influence fear directly, because
what the subjects saw in all three conditions was now the same.
The more natural locomotion techniques resulted both in higher presence scores and in
higher fear scores, indicating that presence has a positive effect on fear. The simulator
sickness was higher for the more natural locomotion techniques, and the results suggest that
this is mainly attributable to the headtracking used to determine the rotation of the viewpoint.
These results are in line with previous research, and can be explained by the fact that nobody
actually keeps his or her head perfectly still, resulting in continuous movement of the
viewpoint, whereas in the trackball condition one simply does not have to touch the trackball
to keep still. The continuous movement in the headtracking conditions will confront the user
more with the latency of the system and will therefore lead to more sensory conflict and thus
to more simulator sickness.
The overall scores of the subjects concerning the usability of the system was not different
between the devices, indicating that subjects found all techniques equally easy to use.
This experiment suggests that more natural locomotion techniques can contribute to higher
levels of presence and fear in a VE, and that all techniques offer comparable levels of control
over what the patient sees and hears in the HMD. The result on simulator sickness suggest
that adding headtracking increases simulator sickness, but more natural interaction
techniques such as walk-in-place do not add further to simulator sickness. In all, the most
natural locomotion technique seems to be the one most suitable for VRET.
5.6 Discussion
The first two experiments showed that presence does not differ between the patient control or
therapist control. However, the second experiment showed that the fear that a patient
experiences in a VE is greater when the patient does not have control over his or her
position. According to our presence model described in the previous chapter, there are two
ways in which the interaction technique can influence the fear of a patient in VRET: by
changing what the patient sees or by changing the sense of presence of a patient. Our results
suggest that when the patient has control, he or she probably finds some way to avoid fearful
situations and thus effect his or her fear. Compared to this effect, the effect of the locomotion
technique on presence seems to be minimal.
107

Chapter 5
The third experiment clearly confirms that, when a virtual fearful situation can be avoided,
people with fear of heights will indeed show avoidance behavior in a VE, and this avoidance
behavior will lead to a significant reduction in that person's fear. This experiment also
showed that, when avoidance behavior is equal, presence does correlate with fear and that
more natural locomotion techniques not only lead to more presence but also to more fear.
There apparently is very little difference in the general usability of the different locomotion
techniques, or even between patient control and therapist control. Also, there does not seem
to be much difference in simulator sickness, although headtracking does seem to add to
sickness reported by subjects.
It should be noted however that only a limited number of locomotion techniques has been
tested in these experiments. Therapist control was only compared to one type of patient
control, and our own research has shown there to be differences between different types of
patient control (i.e. between different types of locomotion technique). Also, the difference
between patient and therapist control was minimized because with either type of control,
patients could use headtracking to look around in the VE.
5.7 Conclusions
Regarding the hypotheses stated in relation to the case of VRET, we can conclude that the
first hypothesis is not correct:
Hypothesis 1: Locomotion controlled by the patient will increase the patient’s sense of
presence.
Both experiment 1 and 2 showed no significant difference between a patient control
condition and a therapist control condition. Similarly, the hypothesis based on this first
hypothesis is also proven incorrect:
Hypothesis 2: Locomotion controlled by the patient will increase the fear a phobic user can
experience.
In fact, experiment 2 showed that fear increases when the patient does not have control over
his of her position in the VE.
Experiment 3 showed that the locomotion technique can have an effect on the presence a
person experiences, thus verifying our next hypothesis:
Hypothesis 3: A more natural locomotion technique for the patient will increase the presence
a user experiences
Also, this experiment showed that fear was effected by the locomotion technique, verifying
the fourth hypothesis:
Hypothesis 4: A more natural locomotion technique for the patient will increase the fear a
phobic user can experience.
In general, based on the three experiments described in this chapter we can conclude that
therapist control seems to be the best type of control over the patient's position, when
considering the effect on the patient's experience.
108

Patient’s user interface
Should however control by the patient seem necessary for other considerations, then more
natural techniques such as the walk-in-place technique seems best suited for VRET.
109

6 Therapist's user interface
The UI for the therapist is in many ways different from the UI for the patient. One reason for
this is that the therapist doesn't have to be immersed in the virtual world. He or she does not
need to have a sense of presence, does not need to feel being inside the virtual reality. Instead
the therapist requires an overview of the situation and needs to be able to give the therapy to
cure the patient. In currently available VR systems for phobia treatment the UI for the
therapist can be called minimal. The therapist is provided with a monitor that shows the same
image as displayed in the HMD, and can control the VE through keyboard commands and
sometimes by using a joystick. Based on our TA we can determine in what way this UI can
be improved, what elements can be added to facilitate the therapist in his or her work. These
changes in the UI can then be evaluated in terms of their usability. For this we will mostly
use subjective evaluation by therapists and, since it is difficult to find actual practicing
therapists who are willing and able to participate in such experiments, by students trained as
therapists. The results reported in this chapter may therefore not be entirely reliable, but
should be seen as indicators.
6.1 Requirements
Our TA showed that the therapist's main goals are to determine the fear the patient is
experiencing, change the experience of the patient and solving any ambiguity that the patient
may have. For the most part, the therapy progresses from determining the fear to changing
the experience and back again. It is these two activities that form the core of the therapist's
work during exposure therapy and that should be facilitated by the UI.
Our TA also showed that the current UI, based on our survey of the current state-of-the-art of
VR phobia treatment systems, can be improved in several areas to achieve a better
facilitation of the therapist work. In short, we identified these areas as:
• Fear determination
• Collaborative steering
• Limited view for navigation
• Frames of reference
• Lack of affordances for the therapist
Any changes to the UI should improve at least one of these areas.
6.2 Design options
One design option already discussed in the previous chapter is whether the patient or the
therapist is controlling the position of the patient in the VE. This choice not only effects the
usability and effectiveness for the patient, it also could have an effect on the way the
therapist performs his or her work. One can imagine that, by relieving the therapist of the
task of positioning the patient in the VE, patient control will make the UI less complicated
for the therapist.
Hypothesis 5: Locomotion controlled by the patient will reduce the complexity of the HCI for
the therapist.
111

Chapter 6
Because the therapist does not have to be immersed in the VR, it seems most feasible to
provide the therapist with a UI similar to traditional interfaces such as the WIMP interface.
The required input and output devices such as monitor, mouse and keyboard are relatively
cheap and the WIMP paradigm is well known to most people, including therapists, making it
easier to learn how to use the system. Of course, some elements of the UI will have to
display 3D graphics, since the VE plays such an important role in the therapy. Hence, we
propose that the therapist's UI take the form of a combined 2D-3D console and that such an
interface will increase the usability of the system.
Hypothesis 6: An extended therapist UI in the shape of a combined 2D - 3D console will
increase the usability of the system.
Figure 6.1 shows a proposition for the design of such a UI. The elements incorporated into
the design are intended to improve the usability of the system. Through use of the familiar
Window Icon Mouse Pointer (WIMP) metaphor the therapist can control the various parts of
the graphical user interface. The two three-dimensional views in the middle of the screen
display the VE from the viewpoint of the patient (top) and from a second, external viewpoint
(bottom). Furthermore, patient name and session-number can be stored with additional
treatment data and a clock is provided to keep track of session time.
Figure 6.1: Overview of the proposed user interface for the therapist.
112

6.2.1 SUDs recording
During therapy, patients are often asked to
report their current level of anxiety on a scale
from zero to ten, where zero means not anxious
at all and ten indicates the highest level of fear
the patient can imagine. Such Subjective Units
of Discomfort (SUDs) are used by therapist to
determine amongst others whether the fear has
diminished during the exposure treatment.
Figure 6.2 show a tool designed to keep track of
these SUDs. By clicking on one of the numbers
Therapist’s user interface
Figure 6.2: Tool for keeping track of
SUDs.
the SUD as well as the time is recorded and displayed either on a chart or in the form of a
table. The back arrow can be clicked to erase the last SUD notation.
Through this tools, the therapist can keep track of the history of SUDs reported by the
patient, making it easier to spot improvement or lack of improvement. We therefore
formulate the following hypothesis:
Hypothesis 6a: Providing a tool for keeping track of the patient’s fear will increase usability
for the therapist.
6.2.2 Free viewpoint
To solve the navigation problems experienced
by some therapists, we propose an external
viewpoint as shown in figure 6.3. This
viewpoint is by default locked on the patient,
showing the viewpoint of the patient in the
shape of a head wearing an HMD.
Also displayed in this external viewpoint is the
location of the railing that is surrounding the
patient in reality. This not only enables the
therapist to determine where the patient is
standing within the railing without looking up,
but it also makes it possible for the therapist to
determine when a virtual railing corresponds to
the real railing, thus creating the illusion that the
patient could physically touch the virtual railing.
In other words, it gives the therapist immediate
insight into the mapping between the real and
Figure 6.3: Extra viewpoint displaying
the VE as well as a representation of the
HMD and the railing surrounding the
patient in reality.
virtual frame of reference of the patient. For instance, in figure 6.3 the virtual railing is not
aligned with the real railing, in figure 6.4 the alignment is perfect. Our hypothesis regarding
this external viewpoint is therefore formulated as follows:
Hypothesis 6b: Providing the therapist with an ‘external’ viewpoint of the VE with a
projection of real world objects will increase usability for the therapist.
113

Chapter 6
To change this external viewpoint the therapist
needs some kind of locomotion control.
Locomotion techniques for desktop VR can
roughly be divided into two categories: “direct
control” and “widget-based control”. Especially
games use the “direct control” method, where an
input device such as a mouse, joystick or
keyboard directly controls viewpoint rotation
and translation. But since the viewpoint is part
of a WIMP interface, it is more feasible to
incorporate a “Widget-based control”. Usually
a window is provided with a bar containing
several special widgets. Through the mouse and
keyboard the user can select and operate a
widget. These controls can offer a wide variety
of locomotion techniques such as gaze directed steering but also discrete selection from a
pop-up menu. The user often is required to indicate the start and stop of the input, for
instance by respectively pressing and releasing the mouse button.
Figure 6.4: External viewpoint where the
real and virtual railing are aligned.
Research has shown these type of interfaces to cause user frustration in a variety of ways,
resulting in several recommendation (Sayers et al., 2000) concerning flexible control of
speed, indication of direction of movement, natural navigation modes, an undo-facility, map
usage, use of self-explanatory buttons and icons and the visibility of the tools available to the
user. Most of these recommendations were incorporated in our design of the therapist UI.
6.2.3 Automatic pilot
Figure 6.5: A part of the two-dimension map (left) and the control for the automatic pilot
(right).
In case the therapist is (also) given control of the position of the patient in the VE, we can
provide the therapist with several controls to manipulate the patient’s location. Beside use of
for instance a joystick, the therapist could use an automatic pilot similar to the one provided
in the current UI where the therapist could move the patient to a location by pressing a button
that corresponded to that specific location in the VE. However, to relieve the therapist of the
task of remembering which button corresponds with which location, we can provide a 2D
map of the VE where these 'navigation points' are displayed. We can state the following
hypothesis:
Hypothesis 6c: Providing the therapist with an autopilot based on a 2D map will increase
usability for the therapist.
114

Therapist’s user interface
Figure 6.5 shows a design of the UI for such an autopilot. In this specific design, the
autopilot can be programmed by dropping a token representing a location in the VE on the
widget of the autopilot panel. By pressing the ‘start’ button the patient will gradually be
moved to the location at the speed set with the speed control. By pressing ‘jump to’ the
patient will immediately be teleported to that location. The location tokens can be picked up
from the two-dimensional map of the VE and could be stored and retrieved from the patient
dossier so the therapist could later return with a patient to a specific situation.
6.2.4 Graphical controls
In current VRET systems, therapist can
sometimes change elements of the VE. The
therapist can do this usually by pressing buttons
on the keyboard. With our combination of a 2D
and 3D UI, we can provide the therapist control
through standard 2D graphical widgets as
displayed in figure 6.6. This shows more clearly
which options are available to the therapist and
how he or she can operate these, and gives the
therapist information about which settings were
selected. We therefore formulate the following
hypothesis:
Hypothesis 6d: Providing the therapist with
graphical controls over specific elements in the
VE will increase the usability for the therapist.
Figure 6.6: Graphical controls for the
therapist to alter elements of the VE.
6.3 Experiment 4: Exploratory evaluation
To gain insight into the relative usability of the UI proposed in the previous paragraph,
typical users should be allowed to try the different design options. In this first experiment we
tested both the old and new UI in a therapy setting.
6.3.1 Method
A user interface was designed to represent the current state-of-the-art as described in chapter
3. This user interface (UI1) and the proposed new design (UI2) were used for exposure
therapy on 27 students diagnosed as having a reasonable amount of acrophobia or
claustrophobia. The therapy was performed by six students of the clinical psychology
department trained in phobia treatment. For acrophobia treatment our VEs of a construction
site, firestairs and rooftop terrace were used in combination with UI1. For claustrophobia
treatment a virtual elevator, closet and narrow hallway were used in combination with UI2.
With this UI, patients were also allowed to move themselves through the VE by use of the
locomotion technique of leaning, already described in previous experiments.
Questionnaires for both ‘patients’ and ‘therapists’ were used to acquire insight into the
overall system usability and the usability of specific elements of the user interface. These
questionnaires can be found in appendix A. The patients were also required to fill in the
Igroup Presence Questionnaire. Additionally, the therapists were interviewed based on video
recordings of the treatment sessions.
115

Chapter 6
6.3.2 Results
In all, three students performing the therapy were interviewed. Two of these preferred UI2 to
UI1. The other student indicated that her preference for UI1 was primarily based on the fact
that the acrophobia worlds provided better possibilities for the exposure therapy; it was
possible to increase the fear for the patient more gradually and to a higher extent.
Interestingly, this student did rate UI2 much higher on our questionnaire with questions
regarding sense of control during the session, ease of use of the system and whether it was
subjectively pleasing to use the system. The other students rated both user interfaces equally.
Analysis of the questionnaires and interviews indicated that several elements of the proposed
design were found helpful in the therapy. The SUDs registration tool was most popular,
receiving 5 out of 5 points both for being used frequently and for easy of use.
The students indicated that the automatic pilot was hardly used at all. This was because the
patient could navigate him or herself through the virtual environment. The students reported
the autopilot to be moderately easy to use, and that they could achieve their goals with the
automatic pilot. Storing certain locations for later use with that patient was never used
because patients were never exposed to the same VE twice.
The external viewpoint was considered useful (average four out of five points). Users
commented that the viewpoint increased their overview of the situation, reduced the need to
look at the patient directly and facilitated moving the patients in tight spots. However, none
of the students ever used the controls to change the viewpoint. The main reason given for this
was that the controls were too complicated. One user commented that there were ‘simply too
many buttons.’
The fact that the patients could navigate themselves in UI2 was evaluated positively by the
therapists (average four out of five points). Due to an error in filling in the questionnaires,
only four of the patients treated with UI1 actually filled in the usability questionnaire,
compared to 14 using UI2. Nevertheless, a one-way-ANOVA showed scores to be
significantly lower (p=.008) for UI2 in response to the question whether the interventions of
the therapists in the controls were considered to be annoying. Also interesting is that subjects
using UI2 scored almost significantly higher on questions regarding how much they looked
around in the VE (p=.058) and the ease with which they could do this (p=.087).
The presence experienced by the patients, as measured with the IPQ, did not differ
significantly between conditions as shown by an ANOVA with condition as independent
factor and IPQ score as dependent factor (F=.173, p=.681, n=27).
6.3.3 Discussion
The fact that the different types of user interfaces were used for treatment of different
phobias made comparison between the systems difficult. As expected, responses showed
usability to be not only a function of the type of user interface but also of the design of the
VE itself. However, qualitative insight was gained into the contribution of several user
interface design elements to system usability. The tool for recording SUDs as well as the
external viewpoint were evaluated positively in terms of their usability. It is also clear that
the controls for changing the external viewpoints need to be simplified before therapists will
use them.
The patient’s locomotion control was considered to enhance usability, both for therapists and
patients. However, this type of interaction had no measurable effect on the sense of presence
that the patients experienced.
116

Therapist’s user interface
The study reported here is, of course, only of an exploratory nature. A second experiment
where the different user interfaces would be used in treatment of one and the same phobia
was planned but not executed due to an unforeseen reduction in our research team.
6.4 Experiment 5: Patient vs. Therapist control
This experiment was combined with experiment 2 described in the previous chapter. Subjects
suffering from fear of flying, including those with underlying complaints of acrophobia and
claustrophobia, were treated in nine different virtual environments. The therapists performed
treatment with both patient control and therapist control, and used a slightly adapted version
of the user interface used in the previous experiment. We could therefore examine the impact
on usability not only of the different control types, but also of the UI in general, similar to
the previous experiment. Furthermore, this experiment provides some insight into the effect
of the controls that are given to the therapist on the fear of the patient and thus on the therapy
itself.
6.4.1 Method
Conditions
The experiment consisted of two conditions: therapist control and patient control. In the
therapist control condition, therapist could change the patient's location in the VE by either
using the joystick or by using a path control as depicted in figure 6.7. Therapists could use
three buttons: 'forward', 'backward' and 'stop' and could set the speed with which subjects
were moved through the VE through a trackbar widget. The path through the VE was
displayed as a red line on the 2D map of the VE.
The path control is a simplification of the autopilot used in the previous experiment and is
based on the observation that therapists always used the preset navigation points and often in
the same order.
Figure 6.7: Path control for the therapist, with the 'forward' button activated.
The control for the external viewpoint was also adapted based on the observations of the
previous experiment that therapist found there to be too many controls. Therapists could now
rotate the viewpoint by selecting and dragging the turn button, and could zoom in and out by
selecting and dragging the zoom button, as shown in figure 6.8.
In the patient condition, therapists had no
control over the patient's location. The therapists
had a total of nine virtual worlds at their
disposal. One of these was the airplane world,
where the patient was always seated and there
therefore was no difference between the two
conditions.
Figure 6.8: Alternative design for the
control of the external viewpoint.
117

Chapter 6
Measures
Therapists were videotaped using the system in both conditions. These videotapes were later
reviewed with the therapists and they were asked to comment on what they were doing and
on their experiences with the system in general. Their comments were noted and analyzed.
Also, therapist were asked to fill in usability questionnaires evaluating the different aspects
of the UI. This questionnaire can be found in appendix A.
Furthermore, all actions by the therapist and patient were automatically recorded by the
system, including location of the patient in the VE and the changing of elements of the VE
by the therapist. Also the SUDs entered by the therapist and the heartrate of the patient were
recorded, as described in the previous chapter.
Subjects
In all, six therapists participated in the experiment, four of whom were students at the
University of Amsterdam trained in the treatment of phobias and two were full therapists at
the VALK foundation. All therapists were female and all therapists were exposed to both
conditions. Prior to the experiment all therapists received instructions regarding the use of
the system, and the system was supplied with a short manual.
6.4.2 Results
Usability
Five of the six therapists reviewed video recordings of treatment sessions. The sixth therapist
had failed to record any of her sessions. During the review of the recording, the most
common remark made by therapists was that the HMD was too heavy and too cumbersome.
Other than that, therapists reported very few usability problems. Two therapists remarked
that in the patient control condition, therapists had very little to do but help the patient
understand the locomotion technique (which was extremely difficult for some patients). One
Element Frequency of use (SD) Ease of use (SD)
2D map 4.5 (.55) 4.3 (.52)
Path display 4.7 (.52) 4.5 (.55)
World controls 4.5 (.84) 4.5 (.55)
Patient’s view 4.7 (.52) 4.5 (.84)
External view 3.3 (.32) 3.8 (.41)
Control for external view 1 (0) 1 (0)
SUDs recording 5 (0) 5 (0)
Path control 4.3 (.82) 4.8 (.41)
Joystick control 1.2 (.41)
Table 6.1: Average scores (and standard deviations) of the different questions regarding
parts of the UI (n=6). Possible scores range from 1 to 5.
therapist remarked that the path display in the 2D map was not always completely clear,
since no directional information was supplied with the path (in other words: which direction
was forward and which was backwards).
All six therapists filled in the usability questionnaire. The results of the first part of the
questionnaire, concerning individual elements of the UI, are displayed in table 6.1.
118

Therapist’s user interface
The 2D map was evaluated positively, and therapists remarked it was useful for determining
where the patient was in relation the whole VE, and how far the patient still had to go, and
was therefore found useful in planning the therapy. Similar remarks were made about the
path display on the 2D map.
The world controls were also evaluated positively. Therapists remarked it was very easy to
understand what elements could be changed in the VE and how to control these elements.
One therapist did remark that it was not always completely clear what the effect would be of
a certain action before the therapist had tried it out once or twice.
The patient’s view was also found to be very useful. Most therapists motivated their
evaluation by saying they needed to know what the patient was seeing. Four therapists
specifically mentioned they found it very useful to see if patients were looking at something
specific in the VE, or were trying to avoid looking at it.
The external view was also used frequently, but less so than the previous elements. Three
therapists specifically mentioned that it was useful to find out if a patient was standing near a
ledge or railing. One therapist mentioned that the second viewpoint allowed her to see what
the patient was doing without looking up. Another therapist mentioned the fact that you
could not actually see the patient’s posture on the screen, only the user’s head., and that she
therefore also still needed to look at the patient.
The controls for the external viewpoint were hardly used at all, and the main reason for this
was, according to all therapists, that the controls were too complicated. Therapists were
uncertain what would happen when they pressed a specific button (even though they all
received specific training on these controls). Also, most therapists remarked that the default
position of the external viewpoint was adequate, so they did not really need to change the
viewpoint.
The SUDs recording tool was used frequently, and therapists had no problem operating it.
The path control was also used often (in the specific condition), and was also found very
easy to use. Very few therapists actually used the joystick, so an evaluation of the joystick’s
ease of use was not possible.
Table 6.2 shows the results of the second part of the usability questionnaire regarding the
evaluation of patient control versus therapist control. One therapist did not have enough
experience with the patient control condition to participate in this comparison of the two
conditions.
Item Score
It was easier for me if the patient had to steer him/herself 2.8 (.84)
Situational awareness
In the therapist control condition, I had less overview over the whole situation 1.2 (.45)
When the patient was navigating, I had more attention for the therapy itself 1 (0)
Complexity
Therapist control is more intensive than patient control 2.2 (1.1)
Therapist control requires more attention than patient control 2 (1.22)
Therapist control is more complicated than patient control 1.4 (.89)
Table 6.2: Average scores (and standard deviation) of the questions regarding the therapist
and patient control conditions (n=5). Possible scores range from 1 to 5.
119

Chapter 6
On the questions regarding the therapist’s awareness of the entire therapy, all therapists
indicated this awareness was higher in the therapist control condition. During this condition,
therapists had to monitor the patient’s fear level very carefully to determine whether the
patient was being moved too fast. In the patient control condition, therapists usually assumed
that patients would more or less set their own pace, and the therapists paid less attention to
their fear levels, resulting in less insight into the course of the therapy.
On the questions regarding the complexity of the UI in the two conditions, the therapist
indicated that patient control was more complex and intensive for the therapist. Three
therapists specifically mentioned that they continuously had to check whether the subject
was going in the right direction. In general, all therapists remarked that the patient
locomotion technique is difficult for many patients, and they are required to put in time and
effort to help the patient navigate through the virtual world.
Four out of five therapists preferred the therapist control condition over the patient control
condition. The other therapist remarked that both types of controls had their advantages, and
patient control was very easy to use when the patient was someone who could easily learn
the locomotion technique.
All therapists preferred path control over joystick control.
Effects of controls
Table 6.3 and table 6.4 show the results of analysis relating the time the patients has spend in
the VE and the progress the patient has made in the VE to either the heartrate 12 or the SUDs
reported by the patient. The results of the linear regression in table 6.3 shows that time has a
negative effect on heartrate, and progress a positive effect. In other words, the fear as
measured by heartrate reduces over time, but increases when a patient moves (or is moved)
further in the VE. The results of the linear regression in table 6.4 shows that for SUDs,
progress also has a positive effect on heartrate. Time, however, does not appear to correlate
with SUDs.
Unstandardized Coefficients
Standardized
Coefficient
B Std. Error Beta t Sig.
(Constant) 19.567 .930 21.040

Therapist’s user interface
Heartrate - baseline SUD
df F Sig. F Sig.
(Intercept) 1 .612 .434 .464 .496
Fasten seatbelt sign 1 62.210

Chapter 6
line with the therapist’s intentions, therapists tend to spend a lot of time and effort helping
the patient in navigating through the VE.
Also, because patient control reduces the therapist’s involvement in determining the progress
through the VE, the determination of the patient’s fear by the therapist is performed less
often and reduces the therapist’s awareness of how the therapy is proceeding.
6.5 Experiment 6: Free viewpoint
In the previously described experiment, the subjective evaluation of the free viewpoint did
not clearly show whether this viewpoint was useful in VRET. Fortunately, the usability of
such a free viewpoint can also be evaluated in a more objective manner. The free viewpoint
was designed to aid the therapist in aligning the real and virtual frame of reference. In an
experiment that was performed concurrently with the previously described experiment, the
contribution of the free viewpoint to the precision and speed with which therapists could
achieve this alignment was evaluated.
The free viewpoint gives the therapist an exocentric view of the situation. Exocentric views
have been known to lead to better global awareness, but can reduce the local awareness
needed for efficient navigation (Lasswell & Wickens, 1995) It is therefore not clear
beforehand whether the free viewpoint will have a positive effect on the therapist’s
performance.
6.5.1 Method
We created a VE with elements typical for phobia treatment: a corridor, a closed elevator and
firestairs. Subjects, who took the role of therapists, were instructed to move a patient
through the VE as fast as possible using the joystick. At certain pre-defined locations they
were asked to place the patient in a position where the patient can look over the virtual
railing and press a button when they thought they had reached this position.
There were two conditions, to which each subject was exposed in random order:
1. The therapist could only see the VE from the patient’s viewpoint
2. The therapist had two displays: one with patient’s viewpoint, one with the free
viewpoint.
Subjects
The ‘therapists’ were nine persons trained in phobia treatment, two of whom were
professional therapists and seven psychology students. They had no prior experience with the
system, apart from a short instruction that was the same for all subjects. The ‘patient’ was the
experiment leader, and did not move unless instructed by the therapist.
Measures
Before the experiment, the subjects filled in the Computer Experience questionnaire (CE), a
five-item questionnaire regarding the subject’s experience with computers and 3D programs.
They also completed a Spatial Ability test, designed to measure the subject’s ability to
perform mental rotations of objects (Neerincx et al., 1999).
During the experiment, the following data was collected:
• Total length of the path taken through the VE.
• Total time taken to complete the VE.
122

Therapist’s user interface
• Total length of the path from certain checkpoints in the VE to where the subject pushed
the button.
• Total time between checkpoint and pressing the button.
• Number of times the therapist made a frontal collision with an object in the VE.
• Total positioning error, which is the difference between the position where the therapist
pushed the button and the ideal position. The total error is the sum of this measure over
several instances in the VE.
• Total rotation error is the difference between the rotation where the therapist pushed the
button and the ideal rotation (parallel to the railing)
After the experiment, subjects completed a usability questionnaire with five five-point Likert
scale questions, that can also be found in appendix A.
6.5.2 Results
First of all, extreme values
were removed from the
data 13 . Neither Spatial
Awareness nor Computer
Experience correlated significantly
with any of the
dependent variables and
were thus ignored in the
further analysis.
A multivariate ANOVA
for repeated measures was
performed with condition
as within-subjects factor and order as
between-subjects factor and the
various measures displayed in table
6.6 as dependent variables. The total
path length was found to be
significantly higher in condition 2
where the subjects had both patient
and external viewpoint at their
disposal. The total positioning error
however was significantly higher in
condition 1.
Apart from the main effects
displayed in table 6.6, an interaction
effect was found between condition
and order on the variable ‘total time
taken’ (F=21.552, p=.002). As
displayed in figure 6.9, subjects only
took longer in condition 2 if they
experienced this condition first.
Measure F p
Total length 6.742 .036
Total time taken 29.534 .001
Length between checkpoints and buttonpress .779 .407
Time between checkpoints and buttonpress 3.320 .111
Number of collisions .726 .422
Total positioning error 10.733 .014
Total rotation error .046 .836
Table 6.6: ANOVA for repeated measures with condition as
within-subjects factor and order as between-subjects factor
(n=9).
Time (milliseconds)
300000
280000
260000
240000
220000
200000
180000
160000
140000
120000
1,2
4
Order
2,1
Condition 1
Condition 2
Figure 6.9: Boxplot of the total time taken to
complete the tasks in the VE in the different
conditions and in the different orders in which
subjects were exposed to the conditions.
13 Extreme values were considered to be more than two standard deviations above or below
mean
123

Chapter 6
Question Mean (SD) F p
The 2nd viewpoint made executing the tasks easier 3.22 (1.56) 0.205 .665
In condition 2 I was watching the 2nd (external) viewpoint more 3.67 (1.41) 7.834 .027
than the viewpoint of the patient
I found the 2nd viewpoint to be confusing 3.00 (1.50) 0.778 .407
I never rotated or zoomed the 2nd viewpoint 4.11 (1.76) 3.889 .089
The 2nd viewpoint made the situation easier to oversee 3.78 (1.20) 3.370 .109
Table 6.7: Mean (and Standard Deviation) of the scores on the usability questions, and the
results of an ANOVA with order as independent factor and score as depended variable.
Otherwise, the total time taken was the same in both conditions.
An analysis of the scores on the items of the usability questionnaire is shown in table 6.7.
Order was found to have a significant effect on the question regarding whether the subject
watched the free viewpoint more than the patient’s viewpoint in condition 2. Results showed
subjects tended to look more at the second viewpoint when they experienced condition 2
first.
6.5.3 Discussion
The lower positioning error in condition 2 indicates that the free viewpoint helps therapists in
positioning the patient more precisely, thus aligning the two frames of reference better.
The results from the questionnaire showed very little difference in subjective usability
between the two conditions. The scores did show that the controls for the second viewpoint
were hardly used by the therapists.
Based on the increase in path length and time taken, we must assume that the second
viewpoint also has a negative effect on the efficiency with which the therapist can navigate
the patient through the VE. The results show that the total time take is longer in condition 2
only when this condition is experienced first, and this can be explained by considering the
usability questionnaire results that show that subjects tend to look more at the free viewpoint
than the patient’s viewpoint when they experience condition 2 first.
From this we can deduce that, although the free viewpoint is better suited for precise
alignment of the two frames of reference, it is less suited for navigation from point to point.
It seems that when the therapist is first confronted with the patient’s viewpoint only, he or
she will learn to use this viewpoint together with the free viewpoint when both displays are
available, and thus is able to better combine the advantages of both viewpoints. When the
user is immediately confronted with both viewpoints, he or she will rely too much only on
the free viewpoint. It seems likely however that this is only the case for novice users. When
users have sufficient experience, they will probably be able to use both viewpoints in an
optimal fashion.
6.6 Discussion
The explorative experiment 4 indicated that the proposed new UI was preferred by the
therapists over the UI based on the current state-of-the-art. Unfortunately, a more thorough
comparison was planned but not executed. Experiment 5 did show that therapists frequently
used the new aspects of the proposed UI, and found these easy to use. This can be seen as
confirmation that the proposed UI elements add to the systems usability, when compared to
the very minimal UI found in VRET systems today. One exception is the free viewpoint, of
which the results of this experiment show that therapists use it less frequently than other
124

Therapist’s user interface
elements, and find it less easy to use. Especially the controls of the free viewpoint are found
to be too complicated to use.
Experiment 5 also showed that the controls of the therapist can have a significant impact on
the fear of the patient, arguing that the therapist should not be denied these important tools in
controlling the therapy.
Another finding of this experiment is that the therapists strongly prefer therapist control to
patient control. Therapist rate the complexity and intensiveness of the therapist control lower
and rated their overall awareness of the therapy higher when they could control the patient’s
position in the VE.
Experiment 6 showed that the free viewpoint can be effective when measured with more
objective measures. It enables the therapists to position the patient more precisely in the VE,
allowing for a better alignment between virtual and real frame of reference. But depending
on previous experience of the therapist, it might also decrease the efficiency in terms of time
needed to move a patient through the VE. However, training will most likely solve this
problem, and efficiency in moving the patient probably does not have very much therapeutic
value.
This experiment furthermore confirmed that the controls of the external viewpoint are hardly
used at all.
6.7 Conclusions
Experiment 5 clearly showed that, contrary to expectations, therapist control was preferred
by the therapist because they rated it to be less complicated. We therefore need to reject our
hypothesis:
Hypothesis 5: Locomotion controlled by the patient will reduce the complexity of the HCI for
the therapist.
However, we should not rule out the possibility that these results are only due to the specific
locomotion technique used for the patient control condition. Fortunately, this does not seem
likely since experience with various locomotion techniques has shown that users do not
experience more or less usability problems with this technique than with other techniques.
In general, the evaluation of the proposed new UI indicated that the therapists preferred the
new design and showed that therapists used the new elements frequently and found them
usable. Thus, we can accept our general hypothesis:
Hypothesis 6: An extended therapist UI in the shape of a combined 2D - 3D console will
increase the usability of the system.
One of the elements that very clearly was welcomed by the therapists was the SUDs
recording tool. Without exception, all therapists indicated they used it frequently and found it
easy to use. We can therefore accept our sub-hypothesis:
Hypothesis 6a: Providing a tool for keeping track of the patient’s fear will increase usability
for the therapist.
An element not unambiguously evaluated positively by the therapist, but shown to be
effective through an objective evaluation in experiment 6 is the external viewpoint. With it,
125

Chapter 6
therapist can have better insight into the mapping between virtual and real frame of
reference. We can accept the related sub-hypothesis:
Hypothesis 6b: Providing the therapist with an ‘external’ viewpoint of the VE with a
projection of real world objects will increase usability for the therapist.
The autopilot based on a 2D map was found useful by the therapists and was used frequently.
However, it is not very clear whether it is an improvement over the ‘old’ autopilot based on
keyboard commands that is sometimes used in current VRET systems. Our usability
evaluation however did not show users to have the same usability problems as found in our
analysis in chapter 3, such as trouble remembering what button corresponded with which
location. We can therefore accept our sub-hypothesis:
Hypothesis 6c: Providing the therapist with an autopilot based on a 2D map will increase
usability for the therapist.
In a similar vein, therapist evaluated the graphical controls over specific elements of the VE
positively. Unfortunately, a straightforward comparison between graphical controls and
keyboard controls did not take place since no controls over the VE were available in our case
study in chapter 3. Even though in theory graphical controls should be easier to use, we have
no empirical evidence to support or reject our sub-hypothesis:
Hypothesis 6d: Providing the therapist with graphical controls over specific elements in the
VE will increase the usability for the therapist .
126

7 Conclusions
In this dissertation the issue was raised that even though technology is rapidly advancing,
these technologies are not adapted for use by most users. On the border between technology
and users we find the area of HCI, and it is this area that we need to improve to fully exploit
the advantages that new technologies can offer. VR is one example of a technology where
there is yet not enough understanding concerning the design of the necessary HCI, especially
when this technology is used in a cooperative fashion. A specific class of applications
depends heavily on the fact that VR can influence users in a non-cognitive way and also
requires close collaboration between users in a VE. One example of this class of applications
is VRET.
The goal set for this dissertation is to extend knowledge of the design of HCI to be able to
cope with applications such as VRET. VRET was therefore defined as a case, an example of
its class, and by attempting to design the HCI for VRET, we hoped to gain insight into what
changes are needed in the design process to facilitate the design of such applications in an
optimal fashion. These changes took shape in the form of a TA method and a presence model
that have consequently been used to generate design options. In chapter 5 and 6, the usability
of these design options has been evaluated in a number of experiments. In the next few
paragraphs, we will discuss the implication of the results of these experiments for the TA
method and presence model.
7.1 Task Analysis method
Based on our TA, several design recommendations were made regarding the user interfaces
of our VRET system. These recommendations were intended to alter the tasks as executed by
the therapist and patient in such a way that the usability problems detected in our first task
model (TM1) were solved. A second task model (TM2) can be used to describe what the
VRET process looks like after the recommendations have been implemented. In these
paragraphs we will discuss what differences were expected between TM1 and TM2 and what
differences were actually found in our experiments. The expectation was that these
recommendations would lead to an increase in usability, and were therefore formulated as
hypotheses regarding changes to the user interfaces and their consequences for the usability
of the system.
7.1.1 Instructing the computer
The first hypothesis stated that locomotion control by the patient instead of by the therapist
would reduce the complexity of the HCI for the therapist. This hypothesis was based on the
assumption that relieving the therapist of controlling the position of the patient would reduce
the number of tasks for the therapist and therefore the complexity of his or her work. In
terms of the diagram as depicted in figure 7.1, we assumed that removing the task ‘Instruct
computer to move patient’ and its underlying actions would lead to a more simpler overall
task for the therapist. However, experiment 5 showed that this hypothesis was incorrect. It
showed that by removing this task, fulfilling the goal ‘Change exposure’ became more
complicated for the therapist because he or she had to change the patient’s location in the VE
indirectly by collaborating with the patient. Also, our findings indicated that removing the
127

Chapter 7
therapist’s direct control over the patient’s position seemed to lead to less frequent changes
in the patient’s exposure by the therapist. Our analysis of the sequence of events showed that
this changing of the exposure was alternated with determining the patient’s fear, and by
reducing the frequency of one task (the changing of the exposure), the frequency of the other
task (determining the patient’s fear) was also reduced, causing the therapist to be less aware
of the progression of the therapy.
Another (sub) hypothesis stated that graphical controls over elements of the VE would
increase the usability for the therapist. This hypothesis was based on the assumption that
graphical widgets provide the therapists with more affordances than arbitrary keyboard
commands, making it easier for the therapist to extract procedural information concerning
the available functions and their operation. Unfortunately, this hypothesis could not be
accurately tested, since our original case on which TM1 is based did not allow for any
changes of elements of the VR, and a planned empirical comparison could not be executed.
Yet another (sub) hypothesis stated that an external viewpoint with a projection of real world
objects on the virtual world would increase the usability for the therapist. This hypothesis
was based on the assumption that this viewpoint would provide accurate information about
the mapping of the real and virtual frames of reference, and that this viewpoint could provide
the therapist with information about the immediate surroundings of the patient that was not
visible on the limited patient’s viewpoint. In both cases it would provide the task ‘Monitor
patient position’ with more suitable state information. Experiment 6 showed that this
Procedures
Sound
T.Controls
T.Screen
P.Posture
seq.
Inform patient
about VE change
Instruct computer
Instruct computer
to to move patient
alt.
Push joystick Press button
Alter VE
element
Figure 7.1: Decomposition of the task 'Instruct computer' in task model 2.
Monitor patient
position
hypothesis can be accepted. Therapists are better able to align the virtual and real frame of
reference using this external viewpoint, but the results also showed that the number of
collisions with obstacles did not differ when this viewpoint was included, indicating that its
usability in terms of detecting nearby obstacles is limited.
128

7.1.2 Determining the patient's fear
One final (sub) hypothesis
based on the TA stated that a
tool for recording and reviewing
SUDs would increase the
usability of the system. This
hypothesis was based on the
assumption that such a tool
would provide the therapist
with historical data about the
SUDs reported by the patient,
making it easier to spot improvement
or lack of improvement
in the patient’s anxiety.
It would relieve the therapist
of memorizing the state information
by facilitating storage
of that information outside of
the therapist’s head by use of a
cognitive artifact: the SUDs
Goals
Procedures
Sound
T.Controls
T.Screen
P.Posture
Ask patient to to
to to report fear
Cure patient
Conclusions
Determine fear Change exposure Solve ambiguity
Monitor patient
response
recording tool. Responses of the therapists in the various experiments showed that this
hypothesis can be accepted. All therapists gave highest marks for the SUD recording tool on
usability and frequency of use.
sim.
Figure 7.2: Decomposition of the goal 'Determine fear' in
task model 2.
7.1.3 Usefulness of the TA method
Our TA method was used for two purposes: it was used for descriptive purposes to describe
the current VRET process and identify and analyze usability problems in the current way of
working, and it was used for predictive purposes, to predict the result of implementing
specific design recommendations.
Whether our description of the current way of working is accurate and complete is not easily
determinable. However, given the empirical approach used and the classification of events
using the predetermined framework of Speech Act theory, we can argue that the validity of
our TM is better supported than a TM that is based on an ad hoc, more subjective analysis as
used in most TA methods.
With the TA method, we were able to uncover several usability problems in the VRET
process. Questions that would be interesting to ask but impossible to answer are: ‘were all
usability problems found?’ and ‘were all found usability problems really a problem?’
However, our TA method covered areas of the TA domain not covered by any single other
TA method. Several usability problems were related to the interdependencies of these areas,
such as the information needs of tasks that required information sources in the UI, resulting
in insight into usability problems such as the need for information on the two frames of
reference that was not satisfied in the original UI design. We can argue that it is therefore
likely that our method uncovered more usability problems than other TA methods that cover
only a part of the TA domain covered by our method.
Our TA could furthermore be used to generate recommendations for design changes. Some
of these recommendations indeed increased the usability for the users, other recommendations
did not. This however does not immediately invalidate our TA method. In fact, both
129

Chapter 7
the outcomes that were predicted and those that were not could all still be explained within
the framework of our TM, as shown in the previous paragraphs. It is only the predictive
value of our TM that is less reliable, and this can be attributed to the fact that the model is
mostly qualitative in nature and not quantitative, and that therefore the result of opposing
effects cannot be predicted beforehand. Only very little TA methods actually claim to be of a
predictive nature, such as the Keystroke model (Card et al., 1983), and these are only
applicable in a very small domain. One should further note that the usability evaluations
were mostly of a subjective nature and involved mostly students instead of practicing
therapists, and may therefore not be completely accurate.
In general, we can argue that, in the case of VRET, and for similar applications, our TA
method is probably best suited to analyze and understand the task at hand. Proving this
beyond a doubt however seems an impossible task and falls beyond the scope of this thesis.
7.2 Presence model
Therapist-Computer Interaction
Patient-Computer Interactions
View of VE
Presence
Figure 7.3: Overview of the presence model for VRET
Fear Effectiveness
Figure 7.3 shows an overview of the presence model. Applying the presence model in the
design of the HCI for VRET resulted in a number of hypotheses regarding the UI for the
patient and the effect that this UI has on the patient’s presence and fear.
7.2.1 Therapist versus patient control
The first hypothesis stated that control by the patient would lead to higher sense of presence
when compared to control by the therapist. This hypothesis was based on the assumption that
when the patient is given control over the VE, he or she will be more involved in the VE and
we had proposed that involvement is one way in which interaction can lead to presence.
Experiments showed that this hypothesis should be rejected, that patient control does not
lead to a higher sense of presence as measured using a validated presence questionnaire. The
most likely explanation for this is that the differences between therapist and patient control
are too small to have an effect on presence. It should be noted that in both types of control,
the patient could change the rotation of the viewpoint and the translation over a small area.
The added possibility of further translation in the VE through use of a locomotion technique
that is not as natural as the headtracking that is already used apparently does not change the
user’s involvement enough to be measured in our experimental setup.
The second hypothesis stated that patient control would also lead to a higher level of fear in
the patient. This hypothesis was based on the assumption that the first hypothesis is true, and
that the increased sense of presence would lead to higher levels of fear. Since the first
hypothesis was rejected and presence was not different for the two control types, we would
therefore expect that the fear would also not be different. However, the experiments showed
that in fact fear was lower in the patient control condition. The most likely explanation is that
130

Conclusions
patient control allowed the patient to avoid fearful situations, and thus by changing the view
of the VE, could reduce the fear of the patient. Experiment 3 showed that, when possible,
users with a phobia will clearly show avoidance behavior in a VE and that this avoidance
behavior will lead to a reduction in fear. This supports our explanation.
7.2.2 Patient locomotion technique
The third hypothesis stated that a more natural locomotion technique would lead to a higher
sense of presence. This hypothesis was based on the assumption that a more natural
locomotion technique would allow users to more easily form a mental model of the
interaction with the VE, and that this HCI would become ready-to-hand or invisible to the
user. Our experiment showed that this hypothesis was correct, that presence was higher for
the more natural locomotion technique. It should be noted that the avoidance behavior of the
subjects in this experiment was the same for all locomotion techniques. In other words: the
view that the subject had of the VE was the same for all techniques, and therefore did not
influence the fear.
The fourth hypothesis stated that a more natural locomotion technique would also lead to
more fear in a phobic user. This hypothesis was based on the assumption that the previous
hypothesis was correct, and that the increase in presence would lead to higher levels of fear.
This hypothesis was also proven correct: subjects using the more natural techniques reported
higher levels of fear.
7.2.3 Usefulness of the presence model
The fact that the first two hypotheses were rejected does not invalidate our presence model.
The results of the experiments showed that there is too little difference between patient and
therapist control in terms of patient involvement and that we therefore overestimated the
magnitude of the effect of the type of control on the sense of presence. Similarly, avoidance
behavior was found to have a strong effect on the level of fear, and it is the magnitude of this
effect that most likely was underestimated in forming our second hypothesis.
When this avoidance behavior is constant, sufficiently large changes in the locomotion
technique have been shown to lead to a higher sense of presence and to higher levels of fear.
Our presence model therefore remains the same, with the added understanding of the
magnitude that opposing effects can take in determining the fear a patient will experience.
Additionally, experiment 5 showed that when the therapist changes elements of the VE, and
thus the patient’s view of the VE, that this will also lead to difference in the level of fear.
One weak point that remains in our presence model is the assumption that more fear will lead
to a more effective therapy. As discussed in chapter 4, there is some evidence that indicates
that more presence and more fear will lead to a lower number of therapies that fail, indicating
a threshold effect: Therapy can be performed as long as the fear can be increased over a
certain undefined threshold value.
The presence model for the first time unambiguously shows the relationship between
presence and fear, between user interaction and presence and between user interaction and
fear. Our extensive review of presence literature has shown that such a model did not yet
exist, at least not in validated form. The model can be used for applications similar to VRET,
and can form the basis for an overall understanding of how HCI can play an important role in
affecting the user in a non-cognitive way.
131

Chapter 7
7.3 User Interface design for VRET
The TA method and presence model were used
to achieve the first part of the goal of this
dissertation: to propose guidelines for the
Human-Computer Interaction of Virtual Reality
Exposure Therapy Systems for treatment of
phobias to improve the usability and effectiveness
of these systems. The second part of the
research goal, the validation of these guidelines,
took place in the shape of the experiments
described in chapters 5 and 6. Based on these
experiments, we can make the following
conclusions regarding what the user interfaces
should look like for VRET systems:
Guideline 1: Control over the position of the
patient is best performed by the therapist instead
of the patient.
With therapist control, the usability for both the
patient and the therapist is higher. For the
patient, it will lead to higher levels of fear and
presumably to a more effective therapy. For the
therapist, it reduces the complexity of the HCI
and provides the therapist with better overview
over the therapy.
Although less relevant given the above
conclusion, in case the patient should be given
control, results show that:
Guideline 2: The most natural patient locomotion
technique is best suited for VRET.
The most natural locomotion technique can
provide patients with the highest sense of
presence and the highest levels of fear, again
presumably leading to more effective therapy.
Figure 7.4: A patient and the therapist,
using the proposed UI.
Figure 7.5: Overview of the 2D / 3D
interface for the therapist.
Guideline 3: Compared to a minimal UI for the therapist that shows only the patient’s point
of view and accepts only keyboard and joystick commands, an interface based on a
combination of 2D and 3D elements should be preferred.
The UI elements that were incorporated in the latter interface were all considered to add to
the usability of the system according to the subjective ratings of therapists:
Guideline 4: An autopilot based on a path displayed on a 2D map should be provided for the
therapist.
132

Conclusions
Guideline 5: A tool for recording and reviewing SUDs should be provided for the therapist.
Guideline 6: Controls over the VE using graphical widgets should be provided for the
therapist.
An external viewpoint showing a projection of real world objects onto the virtual world
received less subjective appraisal, but was nevertheless shown to be useful in positioning the
patient in the VE when objective measures were used:
Guideline 7: An external viewpoint showing a projection of real world objects onto the
virtual world should be provided for the therapist.
7.4 Limitations
Every research has its limitations, and it is awareness of these limitations that can make the
results useful. One important limitation of this research is that the results of our experiments
were obtained in a laboratory setting. Even though our analysis of the current traditional
phobia treatment practice showed that the VRET performed followed the same pattern as
everyday therapy, the situation was still artificial in several ways. For instance, most
therapists in our studies were psychology students trained in phobia treatment instead of
experienced therapists. Protocols for treatment were more strictly adhered to than might be
expected in the real world of daily practice. The only patients were people who knowingly
volunteered to partake in a scientific experiment and can therefore not be assumed to be
completely representative of the entire population of phobics.
Another limitation of our research is that only the case of VRET has been investigated.
Whether the findings will also hold for other applications cannot be know for sure. Also,
only the effect of the choice of locomotion technique has been investigated in relationship to
the patient’s sense of presence and fear. This provides very little assurance that the presence
model applies to other types of interaction as well.
7.5 Future research
Future research should of course take into consideration and possibly correct for these
limitations. Perhaps VRET can be used in a more realistic setting, and possibly other
applications similar to VRET such as training applications could be investigated to determine
whether findings in this research also applies outside of the domain of VRET.
Furthermore, even though a link between interaction and presence has now been established,
the nature of that relationship is still unclear. The theory, proposed in this dissertation, that
interaction can influence presence by involvement and by forming of a mental map, could
not be adequately tested in our experiments. Future experiments with more successful
manipulations of presence could provide more qualitative information about the ways in
which interaction effects presence.
Also, the effect of other types of interaction than locomotion on presence should be investigated.
For instance, providing users with the possibility to manipulate objects in a VE could
increase the sense of presence without increasing the avoidance behavior of the user and thus
may increase the effectiveness of the therapy.
In this research, the technology has mainly been adapted to fit the existing treatment process.
It would be interesting to investigate whether it is possible to adapt the therapy process to
133

Chapter 7
better make use of the advantages of the technology. For example: currently, fear is
gradually increased during therapy by making what the patient sees more scary. According to
the presence model, it could be equally possible to increase fear by keeping the fearful
stimuli constant and slowly increase the patient’s sense of presence.
7.6 General conclusions
VRET is slowly moving from the researcher's lab into the daily practice of therapists. It is in
times like these that usability issues play an important role in the acceptance of such an
advanced technology in a culture that has traditionally been somewhat conservative in
applying computer technology. In the coming years, many therapists will start working with
VRET systems for the first time, and the recommendations in this thesis could make their
work much easier, and could possibly increase the effectiveness of their therapy. It has
shown how the HCI can be designed while taking the therapy process into consideration.
Describing the therapy process itself also added to our understanding of the way therapists
work in reality. This is significant because an objective analysis of this process until now has
been lacking. By uncovering the intricacies of the therapy process, of which many therapists
probably are not consciously aware, the discussion has been made possible in which way
therapy can be improved further using new and advanced technologies.
In general, the TA method and presence model are small steps towards better HCI design,
especially for a relatively new type of systems, that shows great potential.
134

References
Alcaniz , M. Banos , R.M. Botella, C. Perpina, C., Rey , A. Villa, H. (1998) Virtual Reality
Treatment of Claustropobia, , Behaviour Research & Therapy, Vol.36, No.2, February,
pp.239-246
Amant, R. St. (1999) User Interface Affordances in a Planning Representation, Human-
Computer Interaction, Vol. 14, pp.317-354
American Psychiatric Association. (1994) Diagnostic and statistical manual of mental
disorders, 4 th edition, Washington DC
Annet, J., Duncan, K.D. (1967) Task Analysis and training design, Occupational
Psychology, Vol.41, pp.211-221
Apple (1987) Human Interface Guidelines: the Apple Desktop Metaphor, Reading, MA:
Addison-Wesley
Axelsson, A., Abelin, A., Heldal, I., Schroeder, R., Wideström, J. (2000) Cubes in the Cube;
A Comparison of Collaboration in Virtual and Real Environments, Paper presented at
the Presence 2000 workshop, March 27-28, Delft
Badler, N.I., Hollick, M.J., Granieri , J.P. (1993) Real-time Control of a Virtual Human
Using Minimal Sensors, Presence, Vol.2, No.1, pp.82-86
Bakker, N.H. (2001) Spatial Orientation in Virtual Environments, Ph-D dissertation, Delft
University Press
Bangay, S., Preston, L. (1998) An Investigation into Factors Influencing Immersion in
Interactive Virtual Environments, Riva, G., Wiederhold, B.K., Molinari, E. (Eds.),
Virtual Environments in Clinical Psychology and Neuroscience, Ios Press
Barfield, W., Baird, K.M., Bjorneseth, O. J. (1998) Presence in Virtual Environments as a
function of type of input device and display update rate, Displays, Vol.19, pp.91-98
Barnard, P.J, May, J. (1999) Representing Cognitive Activity in Complex Tasks, Human-
Computer Interaction, Vol.14, pp. 93-158
Benyon, D. (1992) The role of task analysis in system design, Interacting with Computers,
Vol.4,No 1, pp.102-123
Benyon, D. Imaz, M. (1999) Metaphors and Models: Conceptual Foundations of
Representation in Interactive System Development, Human-computer Interaction,
Vol.14, pp.159-189
135

Beyer, H., Holtzblatt, K. (1998) Contextual Design: Defining Customer-Centered Systems,
Morgan Kaufmann Publishers Inc., San Fransisco
Bijl, R.V., Ravelli, A. van Zessen, G. (1998) Prevalence of psychiatric disorders in the
general population: results of the Netherlands Mental Health Survey and Incidence Study
(NEMESIS), Social Psychiatry and Psychiatric Epidemiology, Vol.33, pp.587-595
Biocca, F. (1997). The Cyborg's Dilemma: Progressive Embodiment in Virtual
Environments, Journal of Computer-Mediated Communication, Vol.3, No.2
Bouman, T.K., Scholing A., Emmelkamp, P.M.G. (1992) Anxiety Disorders : A
Practitioner's Guide, John Wiley & Sons
Bowman, D.A. (1999) Interaction Techniques for Common Tasks in Immersive Virtual
Environments, PhD thesis, Georgia Institute of Technology, June
Bowman, D.A., Koller, D., Hodges, L. (1998) A Methodology for the Evaluation of Travel
Techniques for Immersive Virtual Environments, GVU technical report 98-04
Brooks, F. (1992) FinalAnnual Technical Report: Walkthrough Project, Report to National
Science Foundation, TR88-035NC:UNC-CH Dept. of Computer Science
Brooks F.P., Airey, J. Alspaugh, J., Bell, A., Brown, R., Hill, C., Nimscheck, U., Rheingans,
P., Rohlf, J., Smith, D. Turner, D.,Varshney, A.,Wang, Y., Weber, H.,Yuanet, X. (1992)
Six Generations of Building Walkthrough, Final Technical Report to the National Science
Foundation (Number TR92-026), Department of Computer Science, University of North
Carolina at Chapel Hill.
Burnst, C.M. (2000) Navigation Strategies with ecological displays, International Journal of
Human Computer Studies, Vol.52, pp.111-129
Card, S.K., Moran, T.P., Newell, A. (1983) The Psychology of Human-Computer
Interaction, Lawrence Erlbaum Associates
Carlin, A.S., Hoffman, H.G. and Weghorst, S. (1997) Virtual Reality and Tactile
Augmentation in the Treatment of Spider Phobia: A Case Study, Behaviour Research &
Therapy, Vol.35, No.2, pp.153-158
Coble, J.R. North, M.M. North, S.M. (1995) Effectiveness of virtual reality environment
desensitization in the treatment of agoraphobia, International Journal of Virtual Reality,
Vol.1, No.2, pp.25-34
Cohen, D.C. (1997) Comparison of self-report and behavioral procedures for assesing
acrophobia, Behavior Therapy, Vol.8, pp.17-23
Davidson, G.C. Neale, J.M. (1996) Abnormal psychology, Wiley publishing co., 7th edition
136

References
Dietz, J.L.G., Widdershoven, G.A.M. (1991) Speech Acts or Communicative Action?,
Proceedings of the Second European Conference on Computer-Supported Cooperative
Work, Amsterdam, the Netherlands, September 25-27, pp.235-248
Dinh, H.Q., Walker, N., Hodges, L. (1999) Evaluating the Importance of Multi-sensory
Input on Memory and the Sense of Presence in Virtual Environments, Proceedings of the
IEEE Virtual Reality 1999 conference, Houston, March, pp.222-228
Dolby, R.G.A (1996) Uncertain Knowledge: An Image of Science for a Changing World,
Cambridge University Press, Cambridge, England
Ellis, S.R. (1996). Presence of Mind: A reaction to Thomas Sheridan's "Further Musing on
the Psychophysics of Presence", Presence, Vol.5, No.2, pp. 247-259
Emmelkamp, P.M.G., Bruynzeel, M., Drost, L., van der Mast, C.A.P.G. (2001) Virtual
Reality Treatment in Acrophobia: A Comparison with Exposure in Vivo,
Cyberpsychology and Behavior, Vol.4, No.3, June, pp.335-341
Emmelkamp, P.M.G., Krijn, M., Hulsbosch, L., de Vries, S., Schuemie, M.J., van der Mast,
C.A.P.G. (2002) Virtual Reality Treatment versus exposure in vivo: A Comparative
Evaluation in Acrophobia, Behaviour Research & Therapy, Vol.40, No.5, May, pp.509-
516
Farrell, P.S.E., Hollands, J.G., Taylor, M.M., Gamble, H.D. (1999) Perceptual control and
layered protocols in interface design: I. Fundamental concepts, International Journal of
Human-Computer Studies, Vol. 50, pp.489-520
Flach, J.M., Holden, J.G. (1998) the Reality of Experience: Gibson's Way, Presence, Vol.7,
No.1, pp. 90-95
Floris, C.F., Ludlow, J. (1981) Doing and speaking in the office, in: G.Fick & R.Sprague
(eds.) DSS: issues and Challenges, London: Pergamon Press
Foley, J.D., van Dam, A., Feiner, S. (1996) Computer Graphics: Principles and Practice,
Addison-Wesley
Freeman, J., Avons, S.E. (2000) Focus Group Exploration of Presence through Advanced
Broadcast Services, Proc. Of the SPIE, Human Vision and Electronic Imaging, pp.3959-
76
Freeman, J., Avons, S.E., Meddis, R., Pearson, D., IJsselsteijn, W. (2000) Using Behavioral
Realism to Estimate Presence: A Study of the Utility of Postural Responses to Motion
Stimuli, Presence, Vol.9, No.2, pp.149-164
Freeman, J., Avons, S.E., Pearson, D., IJsselsteijn, W. (1999) Effects of Sensory Information
and Prior Experience on Direct Subjective Ratings of Presence, Presence, Vol.8, No.1,
pp.1-13
137

Gabbard, J.L, Hix, D. (1997) A Taxonomy of Usability Characteristics in Virtual
Environments, Final Report to the Office of Naval Research
Gabbard, J.L., Hix, D., Swan II, J.E. (1999) User-Centered Design and Evaluation of Virtual
Environments, IEEE Computer Graphics & Applications, Vol.19, No.6,
November/December, pp.2-10
Gardner, M P. (1985), Mood States and Consumer Behavior: A Critical Review, Journal of
Consumer Research, Vol.12 , December, pp.281-300
Gibson, J.J. (1979) The Ecological Approach to Visual Perception, Haughton Mifflin
Gilkey, R.H., Weisenberger, J.M. (1995) The Sense of Presence for the Suddenly Deafened
Adult, Presence, Vol.4, No.4, pp.357-361
Gray, W.D., Salzman, M.C. (1998) Damaged Merchandise? A Review of Experiments That
Compare Usability Evaluation Methods, Human-Computer-Interaction, Vol.13, pp.209-
261
Habermas, J. (1981) Theorie des kommunikativen Handelns, Erster Band, Suhrkamp Verlag
Frankfurt am Main
Halverson, C. (1994) Distributed Cognition as a Theoretical Framework for HCI: Don’t
Throw the Baby Out With the Bathwater - The Importance of the Cursor in Air Traffic
Control, Technical report 9403, Dep. of Cognitive Science, University of California, San
Diego
Heeter, C. (1992) Being There: The Subjective Experience of Presence, Presence, Vol.1,
No.2, pp.262-271
Hendrix, C., Barfield, W. (1996a) Presence within Virtual Environments as a Function of
Visual Display Parameters, Presence, Vol.4, No.3, pp.274-289
Hendrix, C., Barfield, W. (1996b) The Sense of Presence within Auditory Virtual
Environments, Presence, Vol.5, No.3, pp.290-301
Herz, J., Kroch, A., Palmer, R.G. (1993) Introduction to the theory of neural computation,
Addison-Wesley Publishing Company
Hix, D. Swann II, E., Gabbard, J.L., McGee, M., Durbin, J. King, T. (1999) User-Centered
Design and Evaluation of a Real-Time Battlefield Visualization Virtual Environment,
Proceedings IEEE Virtual Reality ‘99, Houston, Texas: IEEE Computer Society Press,
pp. 96-103
Hodges, L. North, M.M., Strickland, D., Weghorst, S. (1997) Overcoming phobias by virtual
exposure, Communications of the ACM, Vol.40, No. 8, August, pp.34-39
138

References
Hodges, L., Rothbaum, B.O, Kooper, R., Opdyke, D., Meyer, T., de Graaf, J.J., Williford,
J.S. (1994) Presence as the defining factor in a VR application, Technical report GIT-
GVU-94-5, Georgia Institute of Technology
Hoffman, H., Groen, J., Rousseau, S., Hollander, A., Winn, W., Wells, M., Furness, T.
(1996) Tactile Augmentation: Enhancing presence in virtual reality with tactile feedback
from real objects, paper presented at the meeting of the American Psychological Society,
San Francisco, Ca.
Hoffman, H., Prothero, J., Wells, M., Groen, J. (1998) Virtual Chess: Meaning Enhances
Users' Sense of Presence in Virtual Environments, International Journal of Human-
Computer Interaction, Vol.10, No.3, pp.251-263
Holyer, A. (1993) Methods for Evaluating user Interfaces, Technical report CSRP 301,
School of Cognitive and Computing Sciences, University of Sussex
Howarth, P.A., Finch, M. (1999) The Nauseogenicity of two methods of navigating within a
virtual environment, Applied Ergonomics, Vol.30, pp.39-45
Hutchins, E. (1995) Cognitions in the Wild, MA: MIT Press
IJsselsteijn, W.A., de Ridder, H. (1998) Measuring Temporal Variations in Presence, paper
presented at the Presence in Shared Virtual Environments Workshop, University College
London, June, pp.10-11
ISO 9241 (1997) Ergonomic requirements for office systems visual display terminals, Parts
10,11,16 International Standards, Parts 1-9, 12-15, 17, Draft Standards; International
Standards Organisation, Switzerland
Johnson, C. (1998) Why ‘Traditional’ HCI Techniques Fail to Support Desktop VR,
D.F.A.Leevers & I.D. Benest (eds.), The 3D Interface for the Information Worker (IEE),
London, UK
Johnson, H. Johnson, P. (1991) Task Knowledge Structures: Psychological basis and
integration into system design, Acta Psychologica, Vol.78, pp.3-26
Kalawsky, R.S. (1998) New Methodologies and Techniques for Evaluating User
Performance In Advanced 3D Virtual Interfaces, D.F.A.Leevers & I.D. Benest (eds.), The
3D Interface for the Information Worker (IEE), London, UK
Kalawsky, R.S. (2000) The Validity of Presence as a Reliable Human Performance Metric in
Immersive Environments, paper presented at the Presence 2000 workshop, March 27-28,
Delft
Kalawsky, R.S., Bee, S.T., Nee, S.P. (1999) Human Factors evaluation techniques to aid
understanding of virtual interfaces, BT Technological Journal, Vol.17, No.1, pp.128-140
139

Kaur, K. (1998) Designing virtual environments for usability, PhD thesis, Center for HCI
Design, City University, London. June
Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G. (1993) Simulator Sickness
Questionnaire: an enhanced method for quantifying simulator sickness, The International
Journal of Aviation Psychology, Vol.3, No.3, pp.203-220
Kennedy, R.S., Stanney, K.M., Dunlap, W.P. (2000) Duration and Exposure to Virtual
Environments: Sickness Curves During and Across Sessions, Presence, Vol.9, No.5,
pp.466-475
Kim, T., Biocca, F. (1997). Telepresence via Television: Two Dimensions of Telepresence
May Have Different Connections to Memory and Persuasion, Journal of Computer-
Mediated Communication, Vol.3, No.2
Kirwan, B., Ainsworth, L.K. (1992) The Task Analysis Guide, Taylor Francis, London
Krijn, M., Emmelkamp, P.M.G., Biemond, R., de Wilde de Ligny, C., Schuemie, M.J., van
der Mast, C.A.P.G. (in preparation) Treatment of acrophobia in Virtual Reality, Head
Mounted Display versus Cave Automatic Virtual Environment
Krijn, M., Grunenberg, S., Loer, M., Rijs, P., Sievers, R., White, K. (2000) Virtual Reality &
hoogtevrees: een onderzoek naar de relatie tussen presence en angst met behulp van
Virtual Reality, KP/OP-3918, University of Amsterdam, July
Lackner, J.R., DiZio, P. (1998). Spatial Orientation as a Component of Presence: Insights
Gained form Nonterrestrial Environments, Presence, Vol.7, No.2, pp.108-115
Lampton, D.R., Knerr, B.W., Goldberg, S.L. (1994) The Virtual Environment Performance
Assessment Battery (VEPAB): Development and Evaluation, Presence, Vol.3, No.2,
pp.145-157
Lamson, R. (1994) Virtual therapy of anxiety disorders, CyberEdge Journal, Vol.4, No.2,
February, pp.6-8
Laurel, B. (1993) Computers as theatre, Addison-Wesley Publishing
Lessiter, J., Freeman, J., Keogh, E., Davidoff, J.(2000) Development of a New Cross-Media
Presence Questionnaire: The ITC-Sense of Presence, Paper presented at the Presence
2000 workshop, March 27-28, Delft
Lif, M, (1999) User-interface modeling - adding usability to use cases, International Journal
on Human-Computer Studies, Vol.50., pp.243-262
Lombard, M. (2000, September) Resources for the study of presence: Presence explication,
Retrieved September 3, 2000 from the World Wide Web:
http://nimbus.temple.edu/~mlombard/Presence/explicat.htm
140

References
Lombard, M., Ditton, T.(1997) At the Heart of it all: The concept of presence, Journal of
Computer-Mediated Communication, Vol.3, No.2
Lombard, M., Ditton, T. (2000) Measuring Presence: A literature-based approach to the
development of a standardized paper-and-pencil instrument, paper presented at the
Presence 2000 workshop, March 27-28, Delft
MacKenzie, I.S. (1995) Input Devices and Interaction Techniques for Advanced Computing,
W. Barfield, Thomas A. Furness III (eds.), Virtual Environments & Advanced Interface
Design, pp.437-470
Mackinlay, J.D., Card, S.K., Robertson, G.G. (1990) Rapid controlled movement through a
virtual 3D workspace, Computer Graphics, Vol.24, No.4, pp.171-176
Mania, K., Chalmers, A. (2000) A User-Centered Methodology for Investigating Presence
and Task Performance, paper presented at the Presence 2000 workshop, March 27-28,
Delft
Mantovani, G., Riva, G. (1999) "Real" Presence: How Different Ontologies Generate
Different Criteria for Presence, Telepresence, and Virtual Presence, Presence, Vol.8,
No.5, pp.540-550
March, S.T., Smith, G.F. (1995) Design and natural science research on information
technology, Decision Support Systems, Vol.15, No.5, December, pp.251-266
Marsh, T. (1999) Evaluation of Virtual Reality Systems for Usability, Altom, M. and
Williams, M. (eds), Proceedings of CHI’99: Human Factors in Computing Systems,
Extended Abstracts, Doctoral Consortium, Pittsburgh, PA, ACM Press
Marsh, T., Wright, P. (1999) Co-operative Evaluation of a Desktop Virtual Reality System,
Workshop on User Centered Design and Implementation of Virtual Environments, King’s
Manor, University of York
McCartney, S. (2001) ENIAC: The Triumphs and Tragedies of the World's First Computer,
Berkley Publishing Group
McGrew, J.F. (1997) A Comparison of Three Methods of Task Analysis: Cognitive
Analysis, Graph-Matrix Analysis and Self-Organizing Networks, International Journal of
Human-Computer Interaction, Vol.9, No.1, pp.41-57
Meehan, M. (2000) An Objective Surrogate for Presence: Physiological Response, paper
presented at the Presence 2000 workshop, March 27-28, Delft
Militello, L.G., Hutton, R.J.B. (1998) Applied cognitive task analysis (ACTA): a
practitioner’s toolkit for understanding cognitive task demands, Ergonomics, Vol.41,
No.11, pp.1618-1641
141

Mine, M. (1995) Virtual Environments Interaction Techniques, UNC Chapel Hill Computer
Science Technical Report TR95-018
Neerincx, M.A., Pemberton, S., Lindenberg, J. (1999) U-WISH Web usability: methods,
guidelines and support interfaces, Report TM-99-D005, Soesterberg, The Netherlands:
TNO Human Factors.
Nielsen, J. (1993) Usability Engineering, AP Professional
Norman, D.A. (1988) The Psychology of everyday things, New York: Basic Books.
Norman, D.A. (1993) Things that make us smart, Addison-Wesley
North, M.M, North, S.M, Coble, J. (1996a) Application: Psychotherapy, flight fear flees,
CyberEdge Journal, Vol.6, No.1, January, pp.8-10
North, M.M., North, S.M., Coble, J. (1996b) Virtual Reality Therapy: An Innovative
Paradigm, IPI Press
North, M.M., North, S.M. Coble, J. (1998). Virtual Reality Therapy: An Effective Treatment
for Phobias, Virtual Environments in Clinical Psychology and Neuroscience, IOS press
O'Brien, J. Büscher, M., Rodden, T., Trevor, J. (1998). 'Red is behind you': the experience of
presence in shared virtual environments, paper presented at the Workshop on Presence in
Shared Virtual Environments, BT Labs., Ipswich
Orlemans, H., Eelen, P., Hermans, D. (1995) Inleiding tot de gedragstherapie, 5e druk, Bohn
Stafleu Van Loghum - Houten / Diegem
Pausch, R., Proffitt, D., Williams, G. (1997) Quantifying Immersion in Virtual Reality,
Proceedings of the 24th annual conference on Computer graphics and interactive
techniques, ACM press, pp.13-18
Picard, R.W. (1997) Affective Computing, M.I.T. Press, Cambridge, MA
Prothero, J.D., Hoffman, H.G. (1995) Widening the Field of View Increases the Sense of
Presence in Immersive Virtual Environments, Technical Report TR-95-2, Human
Interface Technology Lab
Prothero, J.D., Hoffman, H.G., Parker, D.E., Furness III, T.A., Wells. M.J. (1995a)
Foreground/background manipulations affect presence, Proceedings of the Human
Factors and Ergonomics Society 39th Annual Meeting, October 9-13, 1995, San Diego,
California, USA
Prothero, J.D., Parker, D.E., Furness III, T. (1995b) Towards a Robust, Quantitative Measure
for Presence, Proceedings of the Conference on Experimental Analysis and Measurement
of Situation Awareness, pp.359-366
142

References
Reeves, B., Nass, C. (1996) The Media Equation: how people treat computers, televisions
and new media like real people and places, Cambridge: Cambridge University Press,
1996
Regenbrecht, H.T., Schubert, T.W., Friedman, F. (1998) Measuring the Sense of Presence
and its Relation to Fear of Heights in Virtual Environments, International Journal of
Human-Computer Interaction, 10(3), pp.233-249
Richardson, J. Ormerod, T.C., Shepherd, A. (1998) The role of task analysis in capturing
requirements for interface design, Interacting with computers, Vol.9, pp.367-384
Rivera, K., Cooke, N.J., Rowe, A.L, Bauhs, J.A. (1994) Conveying Emotion in Remote
Computer-Mediated-Communication, Conference companion CHI'94, Boston,
Massachusetts, April 24-28, pp.95-96
Rizzo, A.A., Buckwalter, J.G., Humphrey, L., van der Zaag, C., Bowerly, T., Chua, C.,
Neumann, U., Kyriakakis, C., van Rooyen, A., Sisemore, D. (2000) The Virtual
Classroom: A Virtual Environment for The Assessment And Rehabilitation Of Attention
Deficits, CyberPsychology and Behavior, Vol.3, No.3, pp.483-499
Rothbaum, B.O., Hodges, L.F., Kooper, R., Opdyke, D., Williford, J. and North, M.M.
(1995) Effectiveness of computer-generated (virtual reality) graded exposure in the
treatment of acrophobia. American Journal of Psychiatry, Vol.152, No.4, pp.626-628.
Rothbaum, B.O., Hodges, L., Smith, S., Lee, J.H. (2000). A Controlled Study of Virtual
Reality Exposure Therapy for the Fear of Flying, Journal of Consulting and Clinical
Psychology, Vol.68, No.6, pp.1020-1026
Rothbaum, B.O., Hodges, L., Watson, B.A., Kessler, G.D., Opdyke, D. (1996) Virtual reality
exposure therapy in the treatment of fear of flying: a case report, Behaviour Research &
Therapy, Vol.34, No.5-6, pp.477-481
Sayers, H.M., Wilson, S., Myles, W., McNeil, M.D.J. (2000) Navigation in Non-Immersive
Virtual Environments, Proceedings of the 7 th UK VRSIG Conference, University of
Strathclyde, Glasgow, Scotland, pp.43-53
Scheffer, M.M, (1995) Task Analysis for Interactive System and Service Design, Supplement
to Master Thesis, Eindhoven University of Technology, Department of Phylosophy and
Social Sciences
Schloerb, D.W. (1995) A quantitative measure of telepresence, Presence, Vol.4, pp.64-80
Schubert, T.W., Friedmann, F., Regenbrecht, H.T (1999a). Decomposing the Sense of
Presence: Factor Analytic Insights, presented at the 2nd International Workshop on
Presence, University of Essex, UK, 6-7 April
143

Schubert, T.W., Friedman, F., Regenbrecht, H.T. (1999b). Embodied Presence in Virtual
Environments, Visual Representations and Interpretations, Springer-Verlag, London,
pp.268-278
Schubert, T.W., Regenbrecht, H.T., Friedman, F. (2000). Real and Illusory Interaction
Enhance Presence in Virtual Environments, paper presented at the Presence 2000
workshop, March 27-28, Delft
Schuemie, M.J. (2000) Design of Virtual Reality Exposure Therapy Systems - Task
Analysis, CHI 2000 Extended Abstracts, ACM press, pp.345-346.
Schuemie, M.J., Bruynzeel, M., Drost, L. Brinckman, M., de Haan, G., Emmelkamp,
P.M.G., van der Mast, C.A.P.G. (2000) Treatment of Acrophobia in Virtual Reality: a
Pilot Study, F. Broeckx and L. Pauwels (Eds.), Conference Proceedings Euromedia
2000, May 8-10, Antwerp Belgium, pp. 271-275
Schuemie, M.J., van der Mast, C.A.P.G. (1999) Presence: Interacting in Virtual Reality?,
Proceedings of the Twente Workshop on Language Technology 15, pp.213-217
Schuemie, M.J., van der Mast, C.A.P.G. (2001) VR Testbed Configuration for Phobia
Treatment Research, M.E. Domingo, J.C.G. Cebollada & C.P. Salvador (Eds.),
Proceedings of the Euromedia’2001 Conference, April 18-20, Valencia, Spain, pp.200-
204
Schuemie, M.J., van der Straaten, P., Krijn, M., van der Mast. C.A.P.G. (2001) Research on
Presence in Virtual Reality: A Survey, Cyberpsychology and Behavior, Vol.4, No,2.
April, pp.183-202
Searle, J.R. (1969) Speech Acts, Cambridge: Cambridge University Press
Searle, J.R. (1979) Expression and meaning, Cambridge University Press
Sebillote, S. (1995) Methodology Guide to Task Analysis With the Goal of Extracting
Relevant Characteristics for Human-Computer Interfaces, International Journal of
Human-Computer Interaction, Vol.7, No 4, pp.341-363
Shepherd, A. (1998) HTA as a framework for task analysis, Ergonomics, Vol.41, No.11,
pp.1537-1552
Sheridan, T.B. (1992) Musings on Telepresence and Virtual Presence, Presence, Vol.1,
No.1, pp.120-126
Sheridan, T.B. (1996) Further Musings on the Psychophysics of Presence, Presence, Vol.5,
No.2, pp. 241-246
Sheridan, T.B., (1999) Descartes, Heidegger, Gibson, and God: Towards and Eclectic
Ontology of Presence, Presence, Vol.8, No.5, pp.551-559
144

Slater, M. (1999). Measuring Presence: A Response to the Witmer and Singer Presence
Questionnaire, Presence, Vol.8, No.5, pp 560-565
References
Slater, M. Pertaub, D., Steed, A. (1999). Public Speaking in Virtual Reality: Facing an
Audience of Avatars, IEEE Computer Graphics and Applications, Vol.19, No.2, pp.6-9
Slater, M., Sagadic, A., Usoh, M., Schroeder, R. (1998a) Small Group Behaviour in a Virtual
and Real Environment: A Comparative Study, BT Presence Workshop, Martlesham
Heath, June 11-12
Slater, M., Steed, A. (2000) A Virtual Presence Counter, Presence, Vol.9, No.5, pp.412-434
Slater, M., Steed, A. , McCarthy, J., Maringelli, F., (1998b) The influence of body movement
on subjective presence in virtual environments, Human Factors, Vol.40, No.3, pp.469-
477
Slater, M., Steed, A., Usoh, M. (1993) The Virtual Treadmill: A Naturalistic Metaphor for
Navigation in Immersive Virtual Environments, M. Goebel (ed.), First Eurographics
Workshop on Virtual Reality, pp.71-86
Slater, M., Usoh, M. (1993) Representation Systems, Perceptual Positions, and Presence in
Immersive Virtual Environments, Presence, Vol.2, No.3, pp.221-233
Slater, M., Usoh, M., Chrysanthou, Y. (1995) The Influence of Dynamic Shadows on
Presence in Immersive Virtual Environments, Virtual Environments '95, pp. 8-21
Slater, M., Usoh, M., Steed, A. (1994) Depth of Presence in Virtual Environments, Presence,
Vol.3, No.2, pp.130-144
Slater, M. Wilbur, S. (1997) A Framework for Immersive Virtual Environments (FIVE):
Speculations on the Role of Presence in Virtual Environments, Presence, Vol.6, No.6,
pp.603-616
Smith, S., Duke, D. (1999) Using CSP to Specify Interaction in Virtual Environments,
Technical Report YCS 321
Smith, D.C., Irby, C., Kimball, R., Verplank, B., Harslem, E. (1982) Designing the Star user
interface, Byte, April, pp.242-28
Steed, A.J. (1996) Designing Interaction Within Immersive Virtual Environments, PhDthesis,
University of London
Steed, A.J., Slater, M., Sadagic, A. (1999) Leadership and Collaboration in Shared Virtual
Environments, Proceedings of the IEEE Virtual Reality 1999 conference, Houston,
March 1999, pp.112-115
145

Steuer, J., (1992) Defining Virtual Reality: Dimensions Determining Telepresence, Journal
of Communication, Vol.42, No.4, pp.72-93
Stuart, R. (1996) The Design of Virtual Environments, McGraw-Hill
Sutcliffe, A., Carroll, J.M., (1999) Designing claims for reuse in interactive systems design,
International Journal of Human-Computer Studies, Vol.50, pp.213-241
Sutherland, I. (1963) Sketchpad--a man-machine graphical communication system,
Technical Report 296, Lincoln Laboratory, Massachusetts Institute of Technology
Sutherland, I. (1965) The Ultimate Display, Proceedings of the International Federation of
Information Processing Congress, Vol.2, pp 506-508
Sutherland, I. (1968) A head-mounted three dimensional display, FJCC, Vol.33, pp.757-764
Taylor, F.W. (1911) The Principles of Scientific Management, New York: Harper Bros
Taylor, M.M., Farrell, P.S.E., Hollands, J.G. (1999) Perceptual control and layered protocols
in interface design: II. The general protocol grammar, International Journal of Human-
Computer Studies, Vol.50, pp.521-555
Tellegen, A., Atkinson, G. (1974) Openness to absorbing and self-altering experiences
(“absorption”), a trait related to hypnotic susceptibility, Journal of Abnormal Psychology,
Vol.83, pp.268-277
Thie, S., van Wijk, J. (1998) A General Theory on Presence: Experimental Evaluation of
Social Virtual Presence in a Decision Making Task, paper presented at the Presence in
Shared Virtual Environments Workshop, University College London, 10-11 June 1998
Towell, J., Towell, E. (1997). Presence in Text-Based Networked Virtual Environments or
"MUDS", Presence, Vol.6, No.5, pp.590-595
Usoh, M., Catena, E., Arman, S. Slater, M. (2000) Using Presence Questionnaires in Reality,
Presence, Vol.9, No.5, pp.497-503
van der Mast, C.A.P.G. (1995) Developing Educational Software: Integrating Disciplines
and Media, PhD-dissertation, Delft University of Technology
van der Veer, G.C., van Welie, M. (1999) Groupware Task Analysis, Tutorial Notes for
CHI99 workshop, 16th May, Pittsburgh PA, USA
van Gerwen, L. J., Spinhoven, Ph., Diekstra, R. F. W., Van Dyck, R. (1997) People who seek
help for fear of flying: typology of flying phobics, Behavior Therapy, Vol.28, No.2,
pp.237-251
146

References
van Welie, M. (2001) Task-Based User Interface Design, Doctoral Thesis, SIKS Dissertation
Series No.2001-6
Vaudry, C., Crampes, M. (2001) AMEBICA : An adaptive user interface for process
control., Proceedings of the IHM-HCI 2001, Lille, 10-14 September, Vol.2, pp.17-20
Vicente, K.J., Christoffersen, K., Pereklita, A. (1995) Supporting operator problem solving
through ecological interface design, IEEE Transactions on Systems, Man and
Cybernetics, Vol.25, pp.529-545
Viller, S., Sommerville, I. (1999) Coherence: An Approach to Representing Ethnographic
Analysis in Systems Design, Human-Computer Interaction, Vol.14, pp.9-41
Watts, L.A., Monk, A.F. (1998) Reasoning about tasks, activities and technology to support
collaboration, Ergonomics, Vol.41, No.11, pp.1583-1606
Welch, R.B. (1999) How Can We Determine if the Sense of Presence Affects Task
Performance?, Presence, Vol.8, No.5, pp.574-577
Welch, R.B., Blackmon, T.T. Liu, A., Mellers, B., Stark, L.W., (1996) The Effects of
Pictoral Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective
Sense of Presence, Presence, Vol.5, No.3, pp.263-273
Whitefield, A., Hill, B. (1994) Comparative Analysis of Task Analysis Products, Interacting
with computers, Vol.6, No.3, pp.289-309
Wiederhold, B.K., Davis, R., Wiederhold, M.D. (1998) The Effect of Immersiveness on
Physiology, Virtual Environments in Clinical Psychology and Neuroscience
Wilson, G., Sasse, M.A. (2000) The head or the heart? Measuring the impact of media
quality, Extended Abstracts of CHI 2000, April 1-6, The Hague, The Netherlands,
pp.117-118
Winograd, T. (1987) A Language/Action Perspective on the Design of Cooperative Work,
Human-Computer Interaction, Vol.3, No.1, pp.3-30
Witmer, B.G., Singer, M.J. (1998). Measuring Presence in Virtual Environments: A Presence
Questionnaire, Presence, Vol.7, No.3, pp.225-240
Wolpe, J. (1958) Psychotherapy by reciprocal inhibition, Stanford: Stanford University
Press, p.668
Zahorik, P., Jenison, R.L. (1998). Presence as Being-in-the-world, Presence, Vol.7, No.1,
pp.78-89
Zeltzer, D. (1992). Autonomy, Interaction, and Presence, Presence, Vol.1, pp.127-13
147

Abbreviations
ACTA Applied Cognitive Task Analysis
ANCOVA ANalysis of COVAriance
ANOVA ANalysis Of VAriance
Alt. Alternative
AQ Acrophobia Questionnaire
CAVE Computer Aided Virtual Environment
CE Computer Experience
CSCW Computer Supported Collaborative Work
CTA Cognitive Task Analysis
CTM Cognitive Task Model
CUD Comms Usage Diagram
DC Distributed Cognition
DoF Degrees of Freedom
EID Ecological Interface Design
FOV Field Of View
GFOV Geometric Field Of View
GOMS Goals, Operator, Method and Selection rules
GTA Groupware Task Analysis
GUI Graphical User Interface
HCI Human-Computer Interaction
HMD Head-Mounted Display
HTA Hierarchical Task Analysis
ICS Interacting Cognitive Subsystems
IPQ Igroup Presence Questionnaire
ITC_SOPI ITC Sense Of Presence Inventory
ITQ Immersive Tendencies Questionnaire
IWS Interactive Work System
LPT Layered Protocol Theory
MAD Méthode Analytique de Description
MSQ Motion Sickness tendency Questionnaire
NLP NeuroLinguistic Programming
PGMC Participant, Goal, Medium and Content
PQ Presence Questionnaire
Seq. Sequential
Sim. Simultaneous
SSQ Simulator Sickness Questionnaire
SUD Subjective Unit of Discomfort
TA Task Analysis
TAS Tellegen Absorption Scale
TKS Task Knowledge Structures
TM Task Model
UI User Interface
UIM User Interface Modeling
149

UED User Environment Design
VE Virtual Environment
VR Virtual Reality
VRET Virtual Reality Exposure Therapy
WIMP Windows, Icons, Mouse and Pointer
150

Appendix A: Questionnaires
A.1 Task Analysis
Protocol for the interviews of therapists 14
Introduction
State name, institution and the name of the project (VR-project), as well as our partner, the
University of Amsterdam. Ask whether it is OK to make sound recordings.
Explain course of interview
'The goal of the interview is to gain insight into the current way of performing treatment and
any wishes and requirements for a future VR-system. I am therefore interested in the way
you work, not how people work "in general" or "according to theory". This is because we
want the future system to be used by people like you, and we want it to fit in your daily
practice.'
'The interview will last approximately 15 to 13 minutes.'
Informed consent
'Any information you will provide during the interview will be treated confidentially and will
not be linked to your name but to a number. The sole purpose of the sound recording of this
interview is for evaluation by the researcher.'
General information about the therapist
• 'First, I would like to ask you which phobias you have ever treated.'
• 'How often do you come into contact with such a phobia?'
Select one of the phobias (acrophobia, claustrophobia or agoraphobia)
'For the remainder of the interview, I would like you to consider the questions with relation
to the treatment of ....... phobia.'
Goal tree and current implementation
'I would like to ask you to tell me the steps that such a treatment consists of. What does a
treatment look like? Please start from the moment a patient reports to you with complaints.'
The course of this part of the interview can be controlled with the following questions:
• 'Why?', 'With which purpose?' to determine any higher-level goals and 'How?' to
determine lower level goals and implementation details.
• 'On what do you base such a decision?' and 'When will you do that?' to determine the
information used by the therapist for specific steps.
• When files are mentioned: 'What data is stored in such files?'
14 This protocol is translated from Dutch.
151

Appendix A
Complaints and wishes
'Furthermore I would like to ask you if there are any obstacles in your current way of
working, and possibly any wishes and requirements for a future VR-system.'
• 'What is currently often something that goes wrong?'
• 'What takes up most of your time?'
• 'Imagine the ideal situation where you had infinite time, money and manpower at your
disposal. In what way would you change your treatments?'
• 'The goal of our project is to build a VR system for treating patients with phobias. Can
you think of any wishes you might have for such a system?'
Finish
'That's it. Thank you very much for your cooperation.' Offer a demonstration of the current
system. Refer to the website if they're curious.
A.2 Experiment 1
Computer experience questionnaire 15
1. How often do you use a computer? (select the answer that best applies to you)
a. Daily
b. Several times per week
c. Several times per month
d. Several times per year
2. With which input devices are you familiar? (Multiple options possible)
a. Keyboard
b. Mouse
c. Joystick
d. Touch screen
e. Pen/Stylus
f. Drawing tablet
g. 3D input devices (e.g. Space mouse and trackers)
h. Other ....................................
3. Do you sometimes play 3D computer games (that place you in a virtual world), and if
yes, how often? (select answer that best applies to you)
a. Never
b. Daily
c. Once per week
d. Once per month
e. Once per year
4. In case you have any previous experience with Virtual Reality (VR), please state with
which types of VR systems you have experience and how many times you have used
them?
a. Desktop VR(system with VR on an ordinary monitor), number of times .......
152

Questionnaires
b. System with Head-Mounted Display (as you will be using today), number of times
.......
c. CAVE (system with glasses and projection onto walls), number of times ......
d. Other ................................................., number of times ......
A.3 Experiment 2
Computer experience questionnaire 14
Please rate the following questions on a scale from one to five.
Very bad Very good
1. How do you rate your overall computer skills? 1 2 3 4 5
Never Daily
2. How often do you use a computer? 1 2 3 4 5
Never Daily
3. How often do you play 3D games on your computer? 1 2 3 4 5
Never Daily
4. How often do you use 3D programs (excluding games)? 1 2 3 4 5
Never Often
5. Have you ever used a VR-helmet before? 1 2 3 4 5
Usability questionnaire 15
Please indicate with a number from 1 to 7 whether you agree with the following statements.
1 = Completely disagree
7 = Completely agree
1. I had quickly learned how to move through the virtual world.
2. I found it easy to move myself through the virtual world.
3. It was easy to do the things in the virtual world that I wanted to do.
4. I could do all things that I wanted to do.
5. It was immediately clear what I could and what I couldn't do in the virtual world.
6. When I used the system for the 2 nd time, I still new exactly how it worked.
7. I sometimes lost my orientation in the virtual world.
8. I felt like I was in complete control.
9. I liked using the controls of the system.
10. I would have preferred to have more freedom of movement.
11. I found the system to be easy to use.
12. I have often looked around me in the virtual world.
13. It was easy for me to look around in the virtual world.
14. I hardly moved my head.
15 This questionnaire has been translated from Dutch.
153

Appendix A
A.4 Experiment 3
Computer experience questionnaire 16
Please rate the following questions on a scale from one to five.
Very bad Very good
1. How do you rate your overall computer skills? 1 2 3 4 5
Never Daily
2. How often do you use a computer? 1 2 3 4 5
Never Daily
3. How often do you play 3D games on your computer? 1 2 3 4 5
Never Daily
4. How often do you use 3D programs (excluding games)? 1 2 3 4 5
Never Often
5. Have you ever used a VR-helmet before? 1 2 3 4 5
Usability questionnaire 16
Please indicate with a number from 1 to 7 whether you agree with the following statements.
1 = Completely disagree
7 = Completely agree
1. I had quickly learned how to move through the virtual world.
2. I found it easy to move myself through the virtual world.
3. It was easy to do the things in the virtual world that I wanted to do.
4. I could do all things that I wanted to do.
5. It was immediately clear what I could and what I couldn't do in the virtual world.
6. I sometimes lost my orientation in the virtual world.
7. I felt like I was in complete control.
8. I liked using the controls of the system.
9. I found the system to be easy to use.
10. It was easy for me to look around in the virtual world.
11. It took some time before I completely understood the controls.
154

Questionnaires
A.5 Experiment 4
Usability questionnaire for the 'patients' 16
Please indicate with a number from 1 to 7 whether you agree with the following statements.
1 = Completely disagree
7 = Completely agree
1. I had quickly learned how to move through the virtual world.
2. I found it easy to move myself through the virtual world.
3. It was easy to do the things in the virtual world that I wanted to do.
4. I could do all things that I wanted to do.
5. It was immediately clear what I could and what I couldn't do in the virtual world.
6. When I used the system for the 2 nd time, I still new exactly how it worked.
7. I sometimes lost my orientation in the virtual world.
8. I felt like I was in complete control.
9. I liked using the controls of the system.
10. I would have preferred to have more freedom of movement.
11. I found the system to be easy to use.
12. I have often looked around me in the virtual world.
13. It was easy for me to look around in the virtual world.
14. I hardly moved my head.
Protocol usability interview 'therapists' 17
Video
Watch the video of the treatment sessions. Instruct the subject to comment on the images.
Questions
Ask the subject to answer the following questions using a scale from 1 through 5, where 1
means absolutely no and 5 means absolutely yes. Also ask for a motivation of the answers.
For UI1:
1. Do you often use the autopilot?
2. Is the autopilot easy to use?
3. Do you use the joystick often?
4. Is the joystick easy to use?
5. Did you feel you were in complete control during the session?
6. Is the system easy to use?
7. Did you (subjectively) like working with the system?
For UI2:
1. Did you use the free viewpoint often?
2. Did you like using this viewpoint?
16 This questionnaire has been translated from Dutch.
17 This protocol has been translated from Dutch.
155

Appendix A
3. Were you able to see the things you wanted to see in this viewpoint?
4. Did you use the autopilot often?
5. Did you like using the autopilot?
6. Did you use the button 'Disable patient control' often?
7. Do you think it is a good thing when the patient can walk through the VE him or
herself?
8. Did you store the navigation points often?
9. Did you find it easy to store navigation points?
10. How often did you use the notes field?
11. Was the notes field easy to use?
12. Did you use the SUDs recording often?
13. Did you find the SUDs recording easy to use?
14. Did you use the controls of the virtual world often?
15. Did you find the controls of the virtual world easy to use?
16. Have you ever used the alarm clock?
17. Did you find the alarm clock easy to use?
18. Did you feel you were in complete control during the session?
19. Is the system easy to use?
20. Did you (subjectively) like working with the system?
21. Which of the two UI did you enjoy working with the most?
A.6 Experiment 5
Protocol interview therapists 15
Video observations
'I would like to review with you some vide-recordings where you are using the system. I
would like you to tell me what you are doing on the recordings. Try to give as much
commentary about the use of the system as possible, about things you find unusable or
maybe very useful. Anything that pops into your mind.'
Watch 15 minutes of recording of the subject under condition 1, then 15 minutes of
recording of the subject under condition 2.
Questions
'I would like to read to you some statements regarding the system. I would like you to rate,
for each statement, whether you agree with it on a scale from 1 (completely disagree) to 5
(completely agree). I also would like you to motivate your answer.'
Use a display of the UI to indicate which part you're talking about. If subjects asks what you
mean by 'often', respond with: 'often means more than five times per session.'
(2D map)
1. 'I often used the map.'
2. 'I found the map to be clear and unambiguous.'
(World controls)
3. 'I often used the world controls.'
156

4. 'I found the world controls to be easy to use.'
(Patient's viewpoint)
5. 'I often used the viewpoint of the patient.'
6. ' I could see what I wanted to see on the patient's viewpoint.'
(Free viewpoint)
7. 'I often used the free viewpoint.'
8. 'I could see what I wanted to see on the free viewpoint.'
9. 'I often used the controls of the free viewpoint.'
10. 'I found the controls of the free viewpoint easy to use.'
(SUDs notation)
11. 'I often used the SUDs recording.'
12. 'I found the SUDs recording easy to use.'
(Alarm clock)
13. 'I often used the alarm clock.'
14. 'I found the alarm clock easy to use.'
(Path display)
15. 'I often looked at the path display.'
16. ' I found the path display to be clear and unambiguous.'
(Path control)
17. 'I often used the path control (in the specific condition).'
18. 'I found the path control to be easy to use.'
(Joystick control)
19. 'I often used the joystick control (in the specific condition).'
20. 'I found the joystick control to be easy to use.'
(Patient control)
21. 'It was easier for me if the patient had to steer him/herself.'
(Situational awareness)
22. ' In the therapist control condition, I had less overview over the whole situation.'
23. ' When the patient was navigating, I had more attention for the therapy itself.'
(Complexity)
24. 'Therapist control is more intensive than patient control.'
25. 'Therapist control requires more attention than patient control.'
26. 'Therapist control is more complicated than patient control.'
Questionnaires
'Finally, I have two questions about the system in general:'
27. 'Suppose you had to treat someone in VR again tomorrow. Which type of control would
you choose to use?' (Therapist control or patient control)
157

Appendix A
28. 'If you had to choose between joystick and path control, which would you choose?'
(Joystick or path control)
A.7 Experiment 6
Computer experience questionnaire 17
Please rate the following questions on a scale from one to five.
Very bad Very good
1. How do you rate your overall computer skills? 1 2 3 4 5
Never Daily
2. How often do you use a computer? 1 2 3 4 5
Never Daily
3. How often do you play 3D games on your computer? 1 2 3 4 5
Never Daily
4. How often do you use 3D programs (excluding games)? 1 2 3 4 5
Never Often
5. Have you ever used a VR-helmet before? 1 2 3 4 5
Usability questionnaire 18
In this experiment you have taken part in two conditions:
Condition 1: No 2nd viewpoint
Condition 2: With 2nd viewpoint
Below you'll find some statements. Please rate to what extend you agree with these
statements.
1 = Completely disagree
2 = Somewhat disagree
3 = No opinion
4 = Somewhat agree
5 = Completely agree
1. The 2nd viewpoint made executing the tasks easier
2. In condition 2 I was watching the 2nd (external) viewpoint more than the viewpoint of
the patient
3. I found the 2nd viewpoint to be confusing
4. I never rotated or zoomed the 2nd viewpoint
5. The 2nd viewpoint made the situation easier to oversee
18 This questionnaire has been translated from Dutch.
158

Appendix B: Information
needs
Element:
Ask patient to report fear
Description:
Verbal question by the therapist to the patient to report what level of fear he or she
is currently experiencing.
Procedural information needs:
Sources:
State information needs:
Sources:
Element:
Monitor patient response
Description:
Monitoring the reactions of the patients that might indicate his or her fear level
Procedural information needs:
Sources:
State information needs:
Patient’s fear responses
Sources:
Sound
T.Screen
P.Posture
Element:
Instruct patient to look
Description:
Verbally instructing the patient to look into a certain direction
Procedural information needs:
Sources:
State information needs:
Sources:
159

Appendix B
Element:
Instruct patient to change posture
Description:
Verbally instructing the patient to change his or her posture in a certain way
Procedural information needs:
Sources:
State information needs:
160
Sources:
Element:
Monitor patient
Description:
Monitoring whether the patient is following instructions
Procedural information needs:
Sources:
State information needs:
Current patient looking direction
Current patient posture
Current patient location in VE
Potential patient movement
Sources:
Sound
T.Screen
P.Posture
Element:
Inform patient about VE changes
Description:
Verbally informing the patient about any changes that the therapist will make to the
VE
Procedural information needs:
Sources:
State information needs:
Element:
Push joystick
Description:
Sources:
Operation of the joystick in order to move the patient
Procedural information needs:
Operation of the joystick
State information needs:
Sources:
Training
Manual
Sources:

Element:
Press button
Description:
Pressing one of the buttons to start or stop the autopilot
Procedural information needs:
Relationship
locations
between buttons and
Operation of the autopilot
State information needs:
Sources:
Training
Manual
Sources:
Element:
Monitor patient position
Description:
Monitoring the effect that the therapist’s controls have on the VE
Procedural information needs:
Sources:
State information needs:
Position of the patient in the VE
Effect of interactions by patient
Sources:
T.Screen
Element:
Monitor for patient request
Description:
Listening to the patient to determine if the patient has a question
Procedural information needs:
Sources:
State information needs:
Does the patient have a question
Element:
Inform patient about VE
Description:
Informing the patient about the VE
Procedural information needs:
State information needs:
Sources:
Sound
Sources:
Sources:
Information needs
161

Appendix B
Element:
Inform patient of rationale
Description:
Informing the patient about the rationale of the therapy
Procedural information needs:
Sources:
State information needs:
162
Sources:
Element:
Listen for therapist instructions
Description:
Listening for any instructions that the therapist might give
Procedural information needs:
Sources:
State information needs:
Therapist instructions
Element:
Change posture (Change experience)
Description:
Sources:
Sound
Changing one’s posture in accordance with therapist’s instructions
Procedural information needs:
Effect of posture change on the view in
the HMD
State information needs:
Sources:
Training
Sources:
Element:
Monitor VE (Change experience)
Description:
Monitoring the effect that one’s actions have on the VE as seen in the HMD
Procedural information needs:
Sources:
State information needs:
Effect of own actions on VE
Effect of therapist actions
Sources:
Sound
HMD screens

Element:
Report fear level
Description:
Reporting of the fear level to the therapist
Procedural information needs:
State information needs:
Element:
Change posture (Avoid fearful situations)
Description:
Sources:
Sources:
Changing one’s posture to avoid a situation that is fearful to the patient
Procedural information needs:
Effect of posture change on the view in
the HMD
State information needs:
Sources:
Training
Sources:
Element:
Monitor VE (Avoid fearful situations)
Description:
Monitoring the VE to determine whether avoidance behavior is successful
Procedural information needs:
Sources:
State information needs:
Effect of avoidance behavior
Sources:
Sound
HMD screens
Element:
Ask question
Description:
Asking a question to the therapist to resolve an ambiguity
Procedural information needs:
Sources:
State information needs:
Sources:
Information needs
163

Appendix B
Element:
Listen for response
Description:
Listening to any answer the therapist might give in reply to a question
Procedural information needs:
Sources:
State information needs:
Therapists answer
164
Sources:
Sound

Samenvatting
Mens-Machine-Interactie en Aanwezigheid
in Virtuele Realiteit Exposure Therapie
Door technologische ontwikkelingen worden steeds meer geavanceerde gebruikersinterfaces
mogelijk, zoals Virtual Reality (VR). Hierbij wordt de gebruiker blootgesteld aan
driedimensionale computerbeelden en –geluid en wordt de illusie gewekt dat de gebruiker in
een andere wereld is dan de echte, in een virtuele wereld. De gebruiker ervaart dit gevoel van
aanwezigheid, ook wel ‘presence’ genaamd, op een niet-cognitief nivo. In het vakgebied
mens-machine-interactie ontstaat langzaam het besef dat het niet-cognitieve effect dat het
systeem op de gebruiker kan hebben, van groot belang is. Er is een bepaalde klasse van
applicaties waarbij dit effect essentieel is. Hierbij kan men denken aan amusementsprogramma’s
zoals spelletjes, maar ook aan trainingsapplicaties. Bij dit soort applicaties zien
we steeds vaker dat ze door meerdere mensen tegelijk worden gebruikt, en dat deze
gebruikers verschillende geavanceerde gebruikersinterfaces tot hun beschikking hebben om
te communiceren met de computer en met elkaar. Om te bepalen hoe men dergelijke
systemen ontwerpt, is in dit proefschrift gekozen voor een specifieke applicatie die tot deze
klasse behoort: Virtual Reality Exposure Therapy (VRET), oftewel exposure therapie in VR.
VRET is de toepassing van VR bij het behandelen van fobieën zoals hoogtevrees,
claustrofobie en vliegangst, waarbij de patiënt langzaam wordt blootgesteld aan steeds
engere virtuele situaties en de angst door habituatie af zal nemen. In de afgelopen jaren heeft
onderzoek over heel de wereld aangetoont dat VRET effectief kan zijn bij de behandeling
van fobieën. Door de gebruikersinterfaces van deze applicatie te ontwerpen, wordt inzicht
verkregen in hoe men dergelijke applicaties kan ontwikkelen. Het doel van dit proefschrift is
dan ook richtlijnen op te stellen voor de mens-machine-interactie van VRET systemen om de
bruikbaarheid en effectiviteit te verhogen.
Om dit te bereiken zijn, naast de bestaande kennis op het gebied van ontwerpen van
interactieve systemen, twee dingen nodig. Als eerst bestaat er nog geen geschikte methode
om het VRET proces in kaart te brengen en wordt in dit proefschrift een dergelijke Taak
Analyse (TA) methode voorgesteld. Ten tweede bestaat er nog geen inzicht in de relatie
tussen de mens-machine-interactie aan de ene kant en de effectiviteit van de behandeling aan
de andere kant. In dit proefschrift wordt een model voorgesteld waar de effectiviteit wordt
gekoppeld aan de angst die de patiënt ervaart en het gevoel van presence.
De voorgestelde TA methode is een combinatie van bestaande methoden. Het Language /
Action Perspective wordt gebruikt om de multi-modale communicatie tijdens een
behandeling te classificeren en te plaatsen in een taak-hierarchie. Vervolgens wordt een
generieke interactie-cyclus gebruikt om de informatiebehoeften van de gebruiker te
analyseren en te bepalen of het systeem aan deze behoeften voldoet. Hierbij wordt
onderscheid gemaakt tussen informatie over de staat van het systeem en informatie over hoe
het systeem bediend moet worden. De eerste soort informatie moet door het systeem via de
verschillende media verschaft worden, de tweede soort kan via handleidingen en trainingen
opgedaan worden. Een bruikbaar systeem verschaft de gebruiker met al deze informatie via
de gebruikersinterface.
165

De TA methode is getoetst door deze te gebruiken in het analyseren van de huidige manier
waarop VRET plaatsvindt. Hiervoor is een laboratorium-opstelling gemaakt die
vergelijkbaar is met de huidige state-of-the-art op het gebied van VRET. In samenwerking
met de Universiteit van Amsterdam is deze opstelling getest bij het behandelen van
hoogtevrees en zijn enkele bruikbaarheidsproblemen aan het licht gekomen. Zo is het
vaststellen van de angst van de patiënt door de therapeut de meest voorkomende aktiviteit
tijdens de behandeling, maar biedt het systeem hiervoor geen ondersteuning. Het
tegelijkertijd navigeren door de virtuele wereld door zowel de therapeut als de patiënt leidt
tot een aantal problemen, en men kan in het beperkte oogpunt dat beide gebruikers tot hun
beschikking hebben niet altijd alle obstakels waarnemen. De therapeut heeft hiernaast geen
inzicht in de verhouding tussen het referentiekader van de patiënt en het referentiekader van
de virtuele wereld. Dit levert vooral problemen op indien er objekten in de realiteit zijn die
overeen moeten komen met virtuele objekten, zoals de reling waarbinnen de patiënt zich
bevindt. Verder verschaft het huidige systeem via de gebruikersinterface zeer weinig
informatie over hoe de therapeut het systeem moet bedienen en is deze dan ook vaak
aangewezen op de handleiding. Andere problemen die tijdens de hoogtevrees-behandelingen
aan bod zijn gekomen zijn het feit dat er enkele patiënten waren die niet genoeg angst
ervaarden waardoor de behandeling niet effectief was, en het feit dat enkele patiënten
misselijk werden in de virtuele wereld. Dit laatste is een bekend fenomeen in VR, en wordt
ook wel Simulator Sickness genoemd.
Om te verklaren hoe mens-machine-interactie kan bijdragen tot een effectievere therapie is
een model opgesteld. In dit model is de effectiviteit direct gerelateerd aan de angst die een
patiënt kan ervaren in de virtuele wereld. Is deze niet genoeg, dan is de behandeling niet
effectief. Angst in een virtuele wereld wordt door twee dingen bepaald: wat de patiënt ziet en
de hoeveelheid presence die de patiënt ervaart. Als de gebruikersinterface de patiënt
verhindert om enge dingen te zien of in staat stelt om enge dingen te vermijden, zal de
patiënt geen angst ervaren. Ook als de gebruikersinterface de patiënt niet het nodige gevoel
van presence verschaft zal de angst minimaal zijn. Om dit gevoel van presence op te wekken
zal de mens-machine-interactie de attentie van de gebruiker moeten richten op de virtuele
wereld en zal deze interactie het voor de gebruiker mogelijk moeten maken een mentaal
model op te stellen van de virtuele wereld.
Uit het presence model volgt dat de de besturing van het oogpunt van de patiënt
waarschijnlijk het beste door de patiënt zelf kan geschieden, aangezien dit zijn of haar
aandacht het meest op de virtuele wereld zal richten en dus het gevoel van presence zal
verhogen. Deze hogere presence moet dan weer leiden tot een hogere angst. Een tweetal
experimenten hebben echter aangetoond dat het verschil tussen besturing door de patiënt en
besturing door de therapeut te klein is om een effect te hebben op het gevoel van presence.
De angst is echter bij patiëntbesturing een stuk lager en dit is waarschijnlijk te verklaren
doordat de patiënt bij zelfbesturing enge situaties meer kan en zal vermijden. Een derde
experiment toonde aan dat gebruikers inderdaad, wanneer dit mogelijk is, enge situaties
zullen vermijden en dat deze vermijding een effect heeft op de angst. Dit experiment toonde
tevens aan dat een meer natuurlijke interactiemethode tot hogere presence leidt en tot meer
angst bij de patiënt. Uit de experimenten bleek verder dat patiënten geen subjectieve
voorkeur hebben voor een bepaalde interactietechniek en dat Simulator Sickness hoger is
indien de interactietechniek gebruik maakt van headtracking (het koppelen van het virtuele
oogpunt van de gebruiker aan de bewegingen van het hoofd van de gebruiker).
Uit de TA methode volgt een aantal bruikbaarheidsproblemen die voornamelijk betrekking
hebben op de gebruikersinterface voor de therapeut. Om deze op te lossen kan het systeem
166

Samenvatting
worden voorzien van een speciaal voor de therapeut ontworpen interface bestaande uit een
combinatie van tweedimensionale en driedimensionale elementen. Verder is het aannemelijk
dat zelfbesturing door de patiënt de mens-machine-interactie voor de therapeut minder
ingewikkeld zal maken dan wanneer de therapeut de patiënt bestuurd. In een tweetal
experimenten bleek dit laatste echter niet het geval te zijn. Besturing door de patiënt zorgt
ervoor dat de therapeut veel aandacht moet besteden aan het in goede banen leiden van de
navigatie van de patiënt. De verschillende elementen van de voorgestelde interface bleken
echter wel bij te dragen tot de bruikbaarheid zoals deze door de therapeuten werd ervaren.
Met name een hulpmiddel om de door de patiënt gerapporteerde angst te registreren werd
door alle therapeuten als positief ervaren. Een automatische piloot gebaseerd op een
tweedimensionale kaart en grafiche bedieningen voor de virtuele wereld werden ook positief
beoordeeld. Een tweede oogpunt, dat de virtuele wereld toont met een projectie van het
gezichtspunt van de patiënt en reële objecten zoals de reling, werd minder goed beoordeeld
door de therapeuten. Een zesde experiment toonde aan, middels objektieve maatstaven, dat
dit oogpunt wel bijdraagt in de nauwkeurigheid waarmee de therapeut de patiënt kan
verplaatsen in de virtuele wereld.
Uit de hierboven beschreven evaluaties blijkt dat in VRET de besturing van de patiënt door
de virtuele wereld het beste door de therapeut kan geschieden en dat een VRET systeem de
therapeut moet voorzien van een gebruikerinterface bestaande uit tweedimensionale en
driedimensionale elementen. Deze evaluaties dienen echter ook om de TA methode en het
presence model te evalueren. De TA methode is in staat gebleken bruikbaarheidsproblemen
te detecteren en kon gebruikt worden om nieuwe ontwerpsvoorstellen te maken en hiermee is
het nut van deze methode aangetoond. Het presence model bleek eveneens nuttig in het
begrijpen van de relatie tussen mens-machine-interactie en de effectiviteit van de
behandeling. De ontwerpvoorstellen die hieruit volgde bleken echter niet allemaal correct,
aangezien de grootte van twee tegenovergestelde effecten niet correct was voorspeld.
Patiëntbesturing in plaats van therapeutbesturing heeft een minimaal effect op het gevoel van
aanwezigheid, maar heeft wel een groot effect op het vermijdingsgedrag van de patiënt en
leidt dus tot een lagere angst.
Martijn Schuemie, 2003
167

Curriculum vitae
Martijn Schuemie was born on the 5th of August 1975 in Vught, the Netherlands. He
obtained his VWO diploma at the Develstein College in Zwijndrecht in 1993 and started
studying business economics at the Erasmus University in Rotterdam. During his studies, he
also performed volunteer work at several youth-organizations, notably Youth Center New
Adventure and the Youth Counsel of Hendrik Ido Ambacht. Martijn graduated in 1998, with
a master thesis on the subject of Associative Conceptual Space, a form of knowledge
representation for information retrieval systems. Immediately after graduation he started his
research at the Delft University of Technology on the subject of human-computer-interaction
and presence in virtual reality exposure therapy. This research resulted, amongst others, in
this dissertation.
169

SIKS dissertations
1998-1 Johan van den Akker (CWI)
DEGAS - An Active, Temporal Database of Autonomous Objects
1998-2 Floris Wiesman (UM)
Information Retrieval by Graphically Browsing Meta-Information
1998-3 Ans Steuten (TUD)
A Contribution to the Linguistic Analysis of Business Conversations within the
Language/Action Perspective
1998-4 Dennis Breuker (UM)
Memory versus Search in Games
1998-5 E.W.Oskamp (RUL)
Computerondersteuning bij Straftoemeting
1999-1 Mark Sloof (VU)
Physiology of Quality Change Modelling; Automated modelling of Quality Change
of Agricultural Products
1999-2 Rob Potharst (EUR)
Classification using decision trees and neural nets
1999-3 Don Beal (UM)
The Nature of Minimax Search
1999-4 Jacques Penders (UM)
The practical Art of Moving Physical Objects
1999-5 Aldo de Moor (KUB)
Empowering Communities: A Method for the Legitimate User-Driven Specification
of Network Information Systems
1999-6 Niek J.E. Wijngaards (VU)
Re-design of compositional systems
1999-7 David Spelt (UT)
Verification support for object database design
1999-8 Jacques H.J. Lenting (UM)
Informed Gambling: Conception and Analysis of a Multi-Agent Mechanism for
Discrete Reallocation.
171

2000-1 Frank Niessink (VU)
Perspectives on Improving Software Maintenance
2000-2 Koen Holtman (TUE)
Prototyping of CMS Storage Management
2000-3 Carolien M.T. Metselaar (UvA)
Sociaal-organisatorische gevolgen van kennistechnologie; een procesbenadering en
actorperspectief.
2000-4 Geert de Haan (VU)
ETAG, A Formal Model of Competence Knowledge for User Interface
Design
2000-5 Ruud van der Pol (UM)
Knowledge-based Query Formulation in Information Retrieval.
2000-6 Rogier van Eijk (UU)
Programming Languages for Agent Communication
2000-7 Niels Peek (UU)
Decision-theoretic Planning of Clinical Patient Management
2000-8 Veerle Coupé (EUR)
Sensitivity Analyis of Decision-Theoretic Networks
2000-9 Florian Waas (CWI)
Principles of Probabilistic Query Optimization
2000-10 Niels Nes (CWI)
Image Database Management System Design Considerations, Algorithms and
Architecture
2000-11 Jonas Karlsson (CWI)
Scalable Distributed Data Structures for Database Management
2001-1 Silja Renooij (UU)
Qualitative Approaches to Quantifying Probabilistic Networks
2001-2 Koen Hindriks (UU)
Agent Programming Languages: Programming with Mental Models
2001-3 Maarten van Someren (UvA)
Learning as problem solving
2001-4 Evgueni Smirnov (UM)
Conjunctive and Disjunctive Version Spaces with Instance-Based Boundary Sets
172

2001-5 Jacco van Ossenbruggen (VU)
Processing Structured Hypermedia: A Matter of Style
2001-6 Martijn van Welie (VU)
Task-based User Interface Design
2001-7 Bastiaan Schonhage (VU)
Diva: Architectural Perspectives on Information Visualization
2001-8 Pascal van Eck (VU)
A Compositional Semantic Structure for Multi-Agent Systems Dynamics.
SIKS dissertations
2001-9 Pieter Jan 't Hoen (RUL)
Towards Distributed Development of Large Object-Oriented Models, Views of
Packages as Classes
2001-10 Maarten Sierhuis (UvA)
Modeling and Simulating Work Practice BRAHMS: a multiagent modeling and
simulation language for work practice analysis and design
2001-11 Tom M. van Engers (VUA)
Knowledge Management: The Role of Mental Models in Business Systems Design
2002-01 Nico Lassing (VU)
Architecture-Level Modifiability Analysis
2002-02 Roelof van Zwol (UT)
Modelling and searching web-based document collections
2002-03 Henk Ernst Blok (UT)
Database Optimization Aspects for Information Retrieval
2002-04 Juan Roberto Castelo Valdueza (UU)
The Discrete Acyclic Digraph Markov Model in Data Mining
2002-05 Radu Serban (VU)
The Private Cyberspace Modeling Electronic
Environments inhabited by Privacy-concerned Agents
2002-06 Laurens Mommers (UL)
Applied legal epistemology; Building a knowledge-based ontology of the legal
domain
2002-07 Peter Boncz (CWI)
Monet: A Next-Generation DBMS Kernel For Query-Intensive Applications
2002-08 Jaap Gordijn (VU)
Value Based Requirements Engineering: Exploring Innovative E-Commerce Ideas
173

2002-09 Willem-Jan van den Heuvel(KUB)
Integrating Modern Business Applications with Objectified Legacy
Systems
2002-10 Brian Sheppard (UM)
Towards Perfect Play of Scrabble
2002-11 Wouter C.A. Wijngaards (VU)
Agent Based Modelling of Dynamics: Biological and Organisational Applications
2002-12 Albrecht Schmidt (UvA)
Processing XML in Database Systems
2002-13 Hongjing Wu (TUE)
A Reference Architecture for Adaptive Hypermedia Applications
2002-14 Wieke de Vries (UU)
Agent Interaction: Abstract Approaches to Modelling, Programming and Verifying
Multi-Agent Systems
2002-15 Rik Eshuis (UT)
Semantics and Verification of UML Activity Diagrams for Workflow Modelling
2002-16 Pieter van Langen (VU)
The Anatomy of Design: Foundations, Models and Applications
2002-17 Stefan Manegold (UvA)
Understanding, Modeling, and Improving Main-Memory Database Performance
2003-01 Heiner Stuckenschmidt (VU)
Ontology-Based Information Sharing in Weakly Structured Environments
2003-02 Jan Broersen (VU)
Modal Action Logics for Reasoning About Reactive Systems
174

Index
Affective computing .............................55
Affordances.....................................28, 58
Agoraphobia ...................................17, 41
Automatic pilot ...................................115
Cognitive artifacts.................................29
Cognitive walkthrough .........................12
Collaborative steering...........................50
Co-operative evaluation........................13
Design methodology...............................7
Distributed Cognition .....................29, 43
Embodied Cognition framework...........59
Exposure Therapy...................................4
In Vitro ...............................................4
In Vivo..........................................4, 41
Frames of reference ..............................51
Free viewpoint ............................113, 122
Functional requirements .......................49
Guidelines and principles......................12
Gulf of evaluation .................................28
Gulf of execution ..................................28
Heuristic evaluation ..............................12
Human Computer Interaction .................1
Igroup Presence Questionnaire (IPQ) ...64
Immersion.......................................56, 57
Informational requirements...................50
Input devices.........................................83
Interaction cycle....................................23
Interactivity...........................................73
Involvement ..........................................57
Language / Action perspective..............24
Locomotion techniques.........................84
Output devices...................................... 84
Perception-action coupling................... 58
PGMC model........................................ 78
Phobias ................................................... 4
Presence............................................ 2, 55
Procedural information......................... 28
Ready-to-hand ...................................... 58
Semidirected interview......................... 30
Simulator sickness.......................... 53, 61
Social phobia........................................ 17
Specific phobia..................................... 17
State information .................................. 29
Subjective Unit of Discomfort.............. 37
Task Analysis ......................................... 8
Task Analysis method .......................... 20
Task Model ........................................... 21
Task Model 1 .................................... 21
Task Model 2 .................................... 21
Telepresence......................................... 56
Think-Aloud protocol........................... 30
Usability ........................................... 1, 11
Usability engineering lifecycle............... 8
User evaluation..................................... 13
Virtual Environment............................... 1
Virtual Reality ........................................ 1
Virtual Reality Exposure Therapy.......... 4
Brief history...................................... 17
State-of-the-art.................................. 32
Virtual Reality testbed.......................... 34
Vividness.............................................. 70
Window, Icon, Menus and Pointer ......... 1
175