Multimodal Semiotics and Collaborative Design

social/communicative practices. Users also often use mediational means as ‘open texts’ and
appropriate them in ways that are not within their normative uses (Jones and Norris 2005: 50)
In order to exemplify my perspective on these issues, I will refer to two studies on mediated
discourse and action by Theo van Leeuwen and Jay Lemke. Van Leeuwen (2005) focuses on a
comparative analysis of a conventional exhibition design with an educational CD-ROM; and
Lemke (2005) studies the relations between virtual places and temporal experience in the VW of
The Sims. Both studies put forward the issue of ‘virtuality’ into MDA’s agenda. Van Leeuwen’s
analysis focuses on ‘reading paths’, or the ways in which the text functions as an environment for a
multiplicity of reading processes, and structures the analysis at two levels: “[v]isual analysis of the
text, to study the environment of the staged, goal-oriented process, and the pathways it allows” and
“observational, ‘ethnographic’ genre analysis of the user’s trajectory, to study actual staged, goaloriented
reading processes, and so access the (usually internalized) generic patterns that inform it”
(van Leeuwen 2005: 85).
It is also important to think about how the concepts of movement and time are experienced in
digitally mediated interaction, as it is not only the (avatars of) visitors but also the co-designers
who traverse these virtual places. Lemke (2005) analyses ‘The Sims’ – a game-oriented VW
platform for avatar-based socializing – by using two key aspects of (bodily) traversals in time and
(real or virtual) space: chronotope – a term borrowed from Russian philosopher, literary critic,
and semiotician M. Bakhtin (1981, 1986) to denote ‘culturally typical movements and pacings along
trajectories of activity’ – and heterochrony – ‘according to which meaningful activities are linked
across timescales by our use of discursive-semiotic artifacts’- (Lemke 2005: 110). Lemke’s
multimodal theory of ‘place, pace and meaning’ uses chronotope and heterochrony as theoretical
and analytical tools for the inquiry of temporal organization of meaningful activities through the
use of semiotic artifacts as mediational means. I find Lemke’s (2005) use of the terms chronotope
and heterochrony particularly helpful in analyzing the spatial practices in virtual places, both in
terms of the potential visitor experience and the co-production practices through which virtual
places are made. As explained in the theoretical framework, the social semiotic lens to multimodal
meaning-making considers space and time as semiotic resources, and explains the semiotic
transformations of/in space and place by analyzing the construction of meaningful spatial practices
through semiotic resources. Lemke (2005: 115-116) puts forward an important framing to the study
of social action in virtual places by focusing on “the dynamics of space and place” which “makes an
immediate connection to time and pace” as semiotic resources for “expressive meaning-making”,
that is the phenomenological and semiotic coupling of bodily and temporal experiences in the
analysis of user interaction with/in virtual places. Lemke’s use of MDA for studying virtual game
spaces is important not only because it defines meaningful action through the use of a
108