Multimodal Semiotics and Collaborative Design

Classical models of sign and the social dimension of sign-making
Here, my aim is to theorize virtual places and artifacts as multimodal sign systems produced to
communicate particular social functions through potential affordances and constraints for social
action. In order to do so, we must first consider how signs are defined by the mainstream semiotic
traditions, and what social characteristics they represent within the socio-semiotic terminology.
The traditional characterization of the sign is formulated as “something that stands for something
else in some respect” (Nöth 1990: 84), which means “a portion of the syntagmatic plane that is
treated as a unity” for social semioticians Hodge and Kress (1988: 262). The meaning potentials of
signs are derived from the possibility of choice on both paradigmatic and syntagmatic
dimensions, which shape the composition of messages by the communicators’ use of available
resources for sign-making (Jewitt 2009). In order to explain the nature of signification, Ferdinand
de Saussure’s dyadic model focuses on grammars and mental references (‘sound images’), and
theorizes the relation between signifier and signified on arbitrariness. For Saussure, the direct
relationship between signifier and signified results from the linguistic system of differences and
oppositions, which define the meaning of a sign by not only in reference to what its signified is, but
also what it is not (Chandler 2007). On the other hand, Charles S. Peirce’s (1998 [1894]) triadic
conception of the sign considers meaning as fundamentally a dialogic process, and not a material
entity or characteristics of the object. Peirce’s triadic model expands the scope of Saussurean
semiotics by introducing three structural variables (object, sign vehicle and interpretant) and their
symbolic, indexical and iconic relations 1 (Chandler 2007). Peirce’s model emphasizes contextual
motivations and semiotic potentials. Although Saussure’s diachronic perspective on semiotic
relations and his interest in studying the internal systematization of language was also influential,
Peirce’s abductive logic on the dynamic, internal and social, relations between signs and their
interpreters had a more seminal effect on social semiotics. In fact, Hodge and Kress’ (1988) sociolinguistic
theory takes Saussure as an “anti-guide” (1988: 20), through which they invert the
prohibitions of (so-called ‘mainstream’) structuralist semiotics and propose and alternative
semiotics which incorporates cultures, societies and politics into the study of meanings and signs 2 .
However, Hodge and Kress’ (1988) social semiotic approach also criticizes Peirce’s triadic model
for being too much internalized, personalized and isolated from social situations. For Hodge and
Kress, the weakness of this model was its presentation as a sole psychological fact, without
explicitly discussing its roots in social processes. Social semiotics attempts to resolve the
ambiguities due to the psychological classification of semiosis by exploring the underlying social
1
Peirce’s three interconnected components of a sign system were: sign vehicle, object and interpretant
2
Although Saussure’s theory also acknowledges the existence of social factors, they are not considered as fundamental
components of sign making but more as supplementary aspects to understand relations between sign-complexes as
grammatical entities (an apt metaphor would be that of a dictionary, instead of a hypertext in use).
42