Multimodal Semiotics and Collaborative Design

to which all the systemic choices in all three functions predispose us” (O’Toole 1994: 163). Such an
understanding would also limit the scope of the analysis of multimodal semiosis since the
motivations, experiences and reflections of the sign makers would not be uncovered. Thus the
social semiotic approach would have been shifting towards its structuralist roots. This framing
cannot sufficiently address - let alone, explain – the domain-specific social and cultural changes
within the socio-semiotic context in time and through the interactions of subjects, including the
researcher’s. Furthermore, it is possible to claim that one cannot have a proper “sign-maker’s
perspective” (Kress, 2010) without studying what the design of virtual places and artifacts mean to
their makers. In other words, multimodal analysis provides a framework to also reflect on the
researcher’s own interpretations and conceptualizations of what the meaning potentials are, and to
do so in a systematized way. However, in a design-oriented perspective to meaning-making, it is
equally important to understand how the designers find and use semiotic resources, make sense of
the affordances of tools, and organize their practices in order to produce meaningful places and
artifacts. In this view, every decision that is made, every obstacle or challenge that is faced, or every
conscious attempt to build meaningful structures of multimodal compositions, i.e. shaping things
in certain ways, contribute to the making of the overall sign system. Certain aspects of these social
practices are conditioned by the availability of semiotic resources as well as the affordances and
constraints of the available mediational means. Therefore, a socio-cultural view to the coproduction
practices should support the socio-semiotic analysis of the multimodal objects of
design. For this purpose, I employ a mixed-method approach in data production, including not
only textual analysis of places but also interviews and observations with designers, and integrate
the socio-cultural perspective of MDA with the systemic functional framework of multimodal
analysis.
Why multimodal social semiotics, and not linguistic analysis
Although the perspective of social semiotics to meaning-making originates from socio-linguistic
theories, contemporary social semiotic frameworks often shift the foci of their analyses towards the
use of various semiotic resources in the making of multimodal phenomena. In this perspective,
while the use of language and linguistic metaphors to describe the objects are still effective, the
main purpose of conducting a socio-semiotic analysis is to uncover the relational structures and
patterns of semiotic associations in the coherent use of socially available modes. Furthermore,
virtual places are not merely linguistic phenomena. It is possible to deduce spatial cues from
people’s conversations and expressions on their social practices in virtual places, while they are
experienced visually and spatially through the use of digital interfaces. Therefore, the study of
making of places, in which mediated social interactions take place, provides the empirical ground
for multimodal analysis.
102