Multimodal Semiotics and Collaborative Design

analyzes school and office buildings to explore the spatial types of framing, and how design
operates in making of semiotic spaces. Van Leeuwen (2005a) outlines five categories (segregation,
permenance, permeability, separation, rhyme) in which multimodal organization of spatial
elements are considered as semiotic work: they are used either to connect or disconnect, to
segregate or separate, or to create similarity and/or contrast. By critically reviewing Edward T.
Hall’s idea of proxemics (Hall 1966), Hodge and Kress prefer to use the term ‘spatial codes’ rather
than ‘proxemic codes’, as it is thought to combine both the material and social dimensions of
signifying systems.
The spatial code and its transformations carry complex social meanings, and the decoding of
the text reveals both general social categories and processes of negotiation in terms of them
(Hodge and Kress 1988: 57).
Furthermore, Kress’ and Lemke’s social semiotic perspectives often include analyses of the
relations between design, body, tools and objects in their transformations). Kress’ emphasis on
everyday practices reminds the ideas of Merleau-Ponty, who aimed at a description of experience
through every day social action and perception, and of body and its interaction with the
environment through a phenomenology of perception.
The organization of multimodal components into the space - whether physical or virtual space or
graphical space of designed images – may orient and direct the interpreters (interactants) towards
particular behaviors (Kress 2010). For Kress, “[a]ll communication is movement” (Kress 2010:
169), and different media have different affordances to facilitate movement, while each multimodal
arrangement provides both ‘curricula’ and ‘pedagogies’ for interpreters to experience. For
instance, he discusses ‘screen’ as a “spatially organized site of display” (Kress 2010: 170) with
specific affordances for digital communication and image-dominated representation. Users (as
interpreters and learners) experience a ‘constant transformative engagement’ through the use of
graphical interfaces; and as a result, their ‘capacities for acting in the world’ may be enhanced.
As a final point, a specific stream in the social semiotic theory deals with issues that emerge with
consideration of places as semiotic contexts. The discourse-analytical approach to the social
semiotic framework is developed by numerous researchers, including the influential
methodological perspective of mediated discourse analysis (MDA). The unit of analysis in MDA
approach is mediated action –social action mediated through the use of cultural tools – that is
realized by the meaning potentials - affordances and constraints- of mediational means, as users
deploy them in their social worlds/situations (Jones and Norris 2005: 5). Ron and Suzie Wong
Scollon’s (2003) concept of ‘geosemiotics’ develops a four-themed framework to understand how
58