Multimodal Semiotics and Collaborative Design

elements of the situation (field, tenor and mode) into account. The role of semiotic relationships is
embedded into the ideational meanings, role relationships into the interpersonal meanings, and
(multi-)modal configurations of resources into the textual meanings. All of these structural
components (of a semiotic situation) contribute to the making of meanings, therefore should be
taken into account in a social semiotic analysis.
The three perspectives above represent various attempts to theorize the social semiotic functions of
sign as in the social domain. On the other hand, virtual places and artifacts present original
affordances and constraints for user interaction, therefore a critical point of view to the linguistic
meta-functions should also be argued for the analysis of virtual place-making as a complex social
phenomenon. Halliday’s view is strictly a linguistic one, focusing on the particular cases of
language learning and use in social contexts. Whereas Kress (2010) and Lemke (2002, 2009)
theorize multimodal phenomena by focusing also on relations between different modes as semiotic
resources. In terms of the central analytical foci, the multimodal theories above are not very
different from Halliday’s initial theory, in that they both focus on the semiotic, social and
structural/organizational aspects of semiotic processes, and consider sign production (design) as
both a product and a process. On the other hand, virtual places and artifacts are designed to be
experienced both spatially and temporally, rather than merely being read or viewed on a graphic
plane. The spatial characteristics of avatar-mediated presence and navigation in virtual space thus
affect the potentials for construction of meaning through interaction. In my view, design of these
specific types of digital environments can be explained with the help of spatial perspectives on the
systemic functional framework, particularly by notions borrowed from architecture and built
spaces. Therefore, I use Michael O’Toole’s (1994, 2004) conceptualization of the social semiotic
perspective to study buildings of architectural design as multimodal sign systems, expressed in the
experiential, interpersonal and textual meta-functions. For O’Toole (1994), the main characteristics
of buildings that differentiate them from the works of contemplative visual arts (such as paintings
or sculptures) are their practical use values, or the signification of their functions in use to be
precise. This aspect refers to their experiential functions. In this view, the interpersonal function is
described by how the building addresses the individuals through its expressive style and visual
language, including the building as a whole and its individual parts in detailed ranks of scale.
Finally, the textual function explains how composition of each spatial element coheres to each
other and the environment in which the building is constructed. In O’Toole’s (2004) view the three
meta-functions present a dynamic and interrelated form of semiosis, the interpretation of which
should follow a hypertextual logic rather than a linear one. As I have explained, I consider the
multimodal compositions of functional elements in design of virtual places and artifacts as a spatial
task, where virtual buildings and their internal parts (interaction spaces, objects and surfaces)
66