Multimodal Semiotics and Collaborative Design

as multimodal orchestrations of these various discursive and material relations through the coproduction
practices in social contexts. Various modes are transformed, translated and
orchestrated as semiotic resources, the multimodal arrangements of which form the products of
designing. The representation of bodily experience is translated and simulated by the mediation of
avatars, which in turn affects the ways in which places and artifacts are constructed. The ways in
which the co-designers reflect on the issue of virtuality is important because their perspective can
provide insights on their semiotic processes, and their ways of interpreting the context of situation.
The analysis shows that the discourses on the signs of realism and virtuality are often interrelated,
and there is a dynamic semiotic interplay between the meaning potentials of specific modes in
design and their makers’ rhetorical intentions.
One particularly interesting way, in which the participants make sense of constructing virtual
places and artifacts –and in general, being present in a virtual world –, is reflected in their
comparisons with the physical reality. As mentioned in the previous analytical chapters, the codesigners
often generate particular discourses on how their avatars mediate their communications
in virtual places and allow them to perform actions that would otherwise be impossible, or rather
unfeasible. Often times during the interviews, the designers mention how being in a virtual
environment is different, both in terms of being unrestricted by physical (so-called RL) constraints
and finding genuine affordances to design multimodal communication environments. Caitlyn
mentions how SL affords the design team to experiment with forms and functions as they want
because the team is not restricted by material limitations, such as logistics, construction costs of
building models or waste of materials. In PAL case, AmyLee and others consistently mention how
they aimed at a virtual place that would look ‘SL-like’, both in terms of its visual form and its
affordances to accommodate avatar interaction. Their explanations and descriptions of places and
artifacts often refer to their more general comments on being in a specific type of environment
where people (both designers and users) can expand the limitations of their imagination and
experience things that are ‘not possible in RL’. In such cases, the so called ‘hyper-real’ physics of SL
(Santos 2009) provides new affordances and constraints for the designers of places, and they build
their multimodal discourses upon these metaphorical frames. I also relate such remarks to the
conceptual separation of the virtual world from the so-called real to Roland Barthes’ (2009 [1957])
idea of the myth as semiotic construct, through which the connotations of virtual and real can be
explained as second-order significations in design concepts.
The social semiotic framework considers the construction of discourses on reality and signs of
realism as social processes based on constant negotiation of power relations and social control
(Kress and van Leeuwen 2001). In this perspective, participants of the communication process seek
to impose their personal definitions and use their interpretations as ‘semiotic navigational devices’
250