Multimodal Semiotics and Collaborative Design

designers’ purposeful orchestration of socially available material and semiotic resources to
communicate indirectly with their users. Each sign system (place and/or artifact) represents
several choices, as they are composed of smaller multimodal compositions (i.e. textures, scripts, 3D
objects), and each multimodal composition contributes to the overall rhetorical intention, that is to
communicate intended experiential, interpersonal and textual meaning potentials.
Contexts and meaning potentials in semiotic systems
As explained earlier, the systemic functionalist perspective of social semiotics focuses on sign
systems and semiotic resources in particular contexts of situation, through which they present
various semiotic potentials for social interaction (Hodge & Kress 1988, van Leeuwen 2005). With
this rather constructivist framework, the aim of social semiotics is to re-frame the interpretative
semiotic analysis by emphasizing social processes, relations and potentialities through which
meaning is generated. Therefore, the social and cultural contexts in which semiotic action takes
place bears significant importance for analytical purposes. Hodge and Kress (1988) focus on the
semiotic dimensions and meaning potentials of the context of communication. They argue the
theoretical breaking point of social semiotics from its structuralist roots by using Voloshinov’s
(1929) description of the ‘utterance as a social phenomenon’. Every semiotic act (verbal or nonverbal)
is placed within a place and a context of social relations (Scollon and Scollon 2003). For the
social semiotic framework, these places connote not only the immediate contexts of use, but also
the diachronic contexts in which meaning potentials are actualized.
Halliday defines the ‘meaning potential’ as “the paradigmatic range of semantic choice that is
present in the system, and to which the members of a culture have access” (1978: 109). Halliday’s
functionalist view is interested in what a sign system can do, or rather what the social actors are
able to do with it by assessing its internal organization and the social functions that it had evolved
to serve. In doing so, Halliday adopts a sociolinguistic perspective, and regards language as
“encoding of a ‘behavior potential’ into a ‘meaning potential’; that is, as a means of expressing what
the human organisms ‘can do’, in interaction with other human organisms, by turning it into what
they ‘can mean’ (Halliday 1978: 21). Halliday’s theory of social semiotics considers language as one
of possible socially contextualized meaning resources, the potentials of which are inherently related
to the context of use. The sociocultural view of communication in systemic functional framework is
based on the approach’s emphasis on the context of situation in communication.
Context is related to “the environment in which a text unfolds” by Halliday and Hasan (1985).
According to their sociolinguistic view of language, notions of context and text have their
theoretical roots in various sources, including Malinowski’s (1923, 1935) ethno-methodological
45