Multimodal Semiotics and Collaborative Design

The multimodal social semiotic approach of this analysis therefore supplements linguistic analysis
with design theory and a place-oriented perspective to meaning-making. Here, the analysis is not
constrained by people’s verbal expressions of spatial experiences, but actually offers a window to
look into their semiotic practices in and with the virtual places; focusing on the making of signsystems
rather than their verbal representations. Therefore, this is also a practice-oriented semiotic
framework. My interest is in the relations and the socio-cultural contexts in which these relations
are constructed. This approach provides not only a socio-cultural supplement to the multimodal
theory, but also a methodological strategy for production and analysis of data in different virtual
social contexts.
In the following sections, I will reflexively discuss these two dimensions of my analytical framing in
detail, and argue how and why I find it useful to combine multimodal analysis with MDA to
support the analysis of virtual place-making.
5.2. Multimodal Analysis
Multimodal analysis builds on the socio-linguistic foundations of social semiotics (Halliday 1978,
2007, Hodge and Kress 1988) in order to develop theoretical and methodological accounts of (1)
special characteristics of visual semiotics, and (2) relations between images and language in textual
analysis (Kress and van Leeuwen 2001). In their multimodal socio-semiotic approach, Kress and
van Leeuwen sets out to create a theory of semiotics that is “appropriate to contemporary semiotic
practice” by analyzing the “specificities and common traits of semiotic modes which takes account
of their social, cultural and historical production” (Kress and van Leeuwen 2001: 4). In Kress and
van Leeuwen’s perspective, two main foci of multimodal analysis are:
1. The semiotic resources of communication, the modes and the media used, and
2. The (multi-layered) communicative practices in which these resources are used
(Kress and van Leeuwen 2001: 111).
As explained in the theoretical framework, the multimodal social semiotic approach considers the
co-production and use of places and artifacts as semiotic domains of communication. It therefore
allows the analysis of collaboratively designed places and artifacts in SL as digitally mediated
multimodal discourses.
The multimodal paradigm critiques the idea that different modes in multimodal texts have strictly
bounded and framed functions in making of meaning; instead “common semiotic principles
operate in and across different semiotic modes” (Kress & van Leeuwen 2001: 2). Meanings are
made not only by using a multiplicity of semiotic resources, but also in a multiplicity of places and
103