Multimodal Semiotics and Collaborative Design

communication as an essential semiotic resource through which modes are assembled and formed
into messages. The sociocultural definition of situation takes three major elements in consideration
in order to describe a circumstance as a particular situation: (a) the object or focus of attention, (b)
resources to be mobilized by participants, and (c) constraints of the context in which
communication takes place (Taylor et al. 2007 [2000]). A situation is constructed within the
constant transformative relationships between these factors in social contexts, and transforms the
context in which it takes place. In other words, our view of reality is thought to be produced,
maintained and transformed by the trajectory of modes we use to communicate, the primarily
effective of which is language we use to refer to co-construct our own social worlds (Griffin 2009).
As discussed above, my use of social semiotics as primary theoretical framework results from the
theory’s rhetorical approach the social contexts of semiosis, and its predisposition to consider the
sign-maker’s perspective in analyzing communication. Consideration of semiotic resources and
meaning potentials as a fundamental constituent of the social semiotic approach has significantly
affected my analytical framework. Semiotization of meaning potentials in collaborative design
activities, and individual interpretations of the co-designers about the affordances and limitations
of SL in reference to their personal life-worlds were among the central topics of inquiry within all
three case studies.
3.2. The multimodal approach to communication
Although the sociolinguistic view of texts consists primarily of verbal expressions in written and/or
speech forms, contemporary theories on multimodal social semiotics consider a wide range of
semiotic resources as potential texts (i.e. Iedema 2003, Kress and van Leeuwen 2001, van Leeuwen
2005, Kress 2010, Lemke 2002, 2005, 2009a, 2009b, O’Halloran 2004, O’Toole 2004). This view
includes language as a semiotic resource, but expands the focus to relations and orchestrations of a
plurality of modes in construction of meaningful sign systems (Jewitt 2009), such as the virtual
places. Designers of virtual places and artifacts in SL often need to assemble various forms images,
text, music, as well as interactive and animated 3D objects in making of mediated places for
meaningful avatar experiences. Therefore, I consider multimodality as a central element of the
theoretical framework in order to describe the organization and use of various design elements as
multimodal compositions form, function and structure.
In this view, modes represent ‘cultural technologies’ for transcription and inscription of semiotic
texts, which have different affordances and limitations depending on the context. For Kress, “mode
is a socially shaped and culturally given semiotic resource for making meaning” (Kress 2010: 80).
Kress frames his definition of mode by the communicative practices that develop in specific
societies by the use of semiotic resources. The choices of what resources will be used as modes, and
47