Multimodal Semiotics and Collaborative Design

Drawing on this point, I also consider co-production in the three case studies as pragmatic
attempts to make functional environments for social action, rather than merely artistic expressions
of their makers’ aesthetic deliberations. In this respect, the socio-semiotic meta-functions refer to
the sign-makers’ collectively negotiated discourses on function, form and structure. In terms of the
systemic functional framework, design of these multimodal digital objects represent the three
meta-functions in terms of their practical functionalities and use-contexts as experiential meaning
potentials, the style, genre and aesthetic qualities as interpersonal potentials, and the ways in
which different digital elements come together to form virtual places and artifacts as textual
potentials for multimodal sign-making.
In general, my observations in the three case studies point towards a socially constructed
relationship between ‘what is designed’ (the multimodal content) and ‘how, where and by whom it
is designed’ (the social context). A systemic functional framework based on mere textual analysis
can only go so far as to answer the first question, and provide limited insights on the other
(Poynton 1993). However, multimodal social semiotics today has a wide variety of approaches to
investigate relations between ‘the social’ and the dynamic power relations, particularly by focusing
on the situatedness of meaning in mediated action (i.e. Scollon 2001, Scollon and Scollon 2003).
In this view, not only people’s actions have semiotic effects, but also the places in which mediated
social actions take place. The MDA and nexus analysis perspective provides analytical insights on
the how and why questions, revealing how the world is experienced from the designers’ points of
view.
Social semiotics of design as construction of a sense of virtual place
As shown in Chapter 3, the social semiotic approach considers design as a reflection of one’s
interest in participating in the world of communication by using socially available semiotic and
material resources in order to represent their rhetorical intentions. The systemic functional
analysis considers design as multimodal orchestration of semiotic resources (van Leeuwen 2005)
as experiential, interpersonal and textual meaning potentials, through which producers of any
semiotic text (including language but also artifacts and places) communicate with their users. In
other words, design transforms rhetorical intentions to ‘semiotically shaped material’ (Kress 2010).
Furthermore, the rhetorical perspective provides a framing of design as both ‘making of things’
and ‘makings of discourses on things’, thus considers the rhetorical processes as the basis for
design processes.
The analysis in SL also suggests that the design activity is based on signification, which results from
framing and representation of certain aspects of reality. Virtual places and artifacts are constructed
249