Multimodal Semiotics and Collaborative Design

Human and non-human factors in social semiotic analysis
For social semioticians such as Lemke (2000), the emergence of meaningful order in complex
social systems should include the perspectives of not only the participating actors but also that of
the observer, and of the eco-social systems in which meaningful interactions occur. We, therefore,
need a theory that explains the meaning-making interdependencies that involve both human and
non-human actors.
A major shortcoming of social science, according to Latour (1997, 2005) is that it often considers
the ‘means’ or ‘tools’ used in construction of messages as mere ‘intermediaries’, which transport
the meaning without transformation, rather than ‘mediators’, which ‘transform, translate, distort,
and modify the meaning’ as they deliver it. This is why semiotics is a necessary step in studying
social construction of meaning, as it allows the researcher to follow the contexts and environments
in which assemblages are made including both natural and material entities (Latour 1997). The
semiotic resources, both in terms of objects/texts and the spatial actions that take place by the use
of them, have social histories that render the dynamic complexity of the actions meaningful. Here
is also where the perspective of affordances is helpful, as it is used to define the meaning and action
potentials constructed by the mediation of objects. Mediated actions are always material processes,
and meanings are results of collective action in these eco-social systems (Lemke 2000). Therefore,
the perspective of material semiotics is helpful in studying qualitative changes in the eco-social
system through semiosis, which is not just a social and cultural practice, but also a material activity
with human and non-human actors (Lemke, 2000). In combination with the meta-functional
analysis of sign-systems, this materialistic perspective to semiotics is important for social semiotic
analysis in order to understand the eco-social dynamics of semiosis and its relations with social
practices. When it is considered as a study of associations (Latour 2005), the ANT approach
provides a relevant background theory to explore the social construction of meaning.
On the other hand, the systemic functional method of social semiotic analysis differentiates from
the study of actor-networks, particularly because of the privileged position of the text as the
domain of meaning potentials. Here, the non-human actors are conceptualized as ‘mediational
means’, through which human actors claim the role of co-designers to produce the virtual places.
The material semiotic approach of ANT focuses on connections in expense of rendering the social
world flat, whereas my perspective distinguishes from ANT in epistemological and methodological
aspects. The flat modeling of social systems in ANT would render them with no distinct levels, but
rather complex intersecting networks of interactional processes. On the other hand, this flat and
heterogeneous model would also emphasize one key notion of mediation: namely, the artifacts
(Lemke 2000). The design and interpretation of meaning potentials in virtual places and artifacts
120