full Paper - Nguyen Dang Binh

Figure 1: Overview of the GRAB system.
ments [12] such as large memory games [14], explorative
games [15] and traf£c environments [16] have been investigated.
At present several commercial 3D Haptic interfaces
are available on the market. Some of them have limitations
when it comes to the realistic exploration of virtual
3D graphics by touch. Force feedback gloves provide direct
feedback independently to multiple £ngers but have limited
degrees of freedom and are only capable of producing small
forces. Being grounded on the hand, they are unable to prevent
movements other than in the £ngers, so there are a number
of effects they cannot produce. Beyond that, in order to
£t with the kinematics of the hand, these exoskeleton devices
usually behave a very poor mechanical stiffness which make
the force feeling at the contact quite unrealistic. Desktop
haptic devices, such as the PHANToM (Sensable Technologies),
are capable of producing better levels of force in three
degrees of freedom, making it possible to realistically represent
3D solid objects. However, the single point haptic interaction
mode has still some constraints for blind and visually
impaired people, due to persistent dif£culties with orientation
(miss a reference point), the spatial memory, locating
objects, staying in touch with objects and perceiving complex
shape and size ([17],[18],[19]). Jansson ([20]) showed
that, for shape and form perception, exploration using a single
£nger falls well short of the utility provided by using ten
£ngers. However, the same research shows that two-£ngered
exploration is signi£cantly better than one-£ngered and not
much worse than ten £ngered. Taking into account the dif-
£culty of providing a multi-£ngered device that is capable
of producing the forces required for realistic 3D haptic rendering,
a two-£ngered device seems like a good direction to
explore. Previous experiences of the partners of the GRAB
consortium showed that the employment of two PHANToMs
devices is not suitable when problems of manipulation and
shape recognition are addressed. The cognitive capacity of
the user in recognizing the shape of the object results reduced
when the users are deprived of the visual and tactile
Massimo Bergamasco / Future trends and Applications, Medicine
53
feedback. Therefore in order to allow visual impaired users
to interact with a virtual environment using only the haptic
and proprioceptive senses, a reconstruction of the environment
with object large enough is required This set of considerations
convinced the partners of the GRAB consortium
that any approach to develop a novel virtual reality system
to enable IST access for visually impaired people could not
rely on existing systems like the above cited. So, they decided
to proceed in the development of a novel system: the
GRAB system
2. The GRAB system
The GRAB system is a new Haptic & Audio Virtual Environment
that allows blind and visually impaired persons
to have access to the three-dimensional graphic computer
world through the sense of touch (using a new dual-£nger
haptic interface) and augmented by audio input and voice
commands.
Instead of displaying just the images of the 3D objects
with a visual display, the new environment allows its user to
feel with his/her £ngers the shape of the virtual 3D objects.
This is achieved using a 3D force-feedback Haptic Interface
speci£cally developed to touch 3D virtual objects both
with the thumb and the index £ngertips or both index £ngertips
while moving the hands in a desktop workspace. As the
user moves their £ngers over the virtual object he/she feels
contact forces at the £ngertips and can recognize its geometric
features (such as corners, surface edges, curvature,..),
distinguish sizes and distances and understand spatial relationships
between elements. During the haptic exploration,
the user can also receive audio messages (speech and nonspeech)
and execute verbal and keyboard commands. The
operator screen renders the virtual scene and what the user is
doing at each moment (position of the user’s £ngers, movement
of any object, ³E.). The £gure 1 shows a work session
with the new GRAB system As it shows the £gure 2, the
new system is based on the integration of three tools:
• A new two-£nger 3D force-feedback Haptic Interface.
• A commercial tool, ViaVoice (IBM) to provide speech
recognition and voice synthesis
• A new Haptic Geometric Modeller to allow the interaction
with any 3D virtual object through haptic stimuli, sounds
aids and speech recognition.
Figure 2: GRAB system architecture.
c○ The Eurographics Association 2005.