Multimodal Semiotics and Collaborative Design

of culture. Finally, mode provides ‘means for realizing/materializing meanings’ with resources that
are appropriated within societies.
Design is considered as a mediating level of practice between content and expression, and defined
by Kress and van Leeuwen (2001: 5) as “(uses of) semiotic semiotic resources, in all semiotic modes
and combinations of semiotic modes.” Design focuses on individuals’ realization of their interests
in participating in the social world in a prospective way. The prospective nature of design results
from its capacity to translate rhetor’s intentions into ‘semiotically shaped material’, and transform
the available resources into “means for action in the designer’s interest” (Kress 2010: 22)..
The political and social interests of the rhetor are the generative origin and shaping influence
for the semiotic arrangements of the designer (Kress 2010: 50).
The process of the rhetor refers to the social dimension and politics of communication, which is
ideally present at all stages of sign-making including the initial idea/concept generation to policy
making and overall shaping of the designed messages (Kress, 2010: 43). Kress’ definition of design
“foregrounds a move away from anchoring communication in convention as social regulation.”
(Kress 2010: 6) Semiotic resources connote potentials, while competences refer to grammars, and
thus, constraints. In co-production of virtual places and artifacts, meaning generation through
design is closely related to the co-designers’ choices to represent certain genre characteristics, or
create new visual syntaxes to emphasize their rhetorical intentions 16 . These differences and
similarities in the use of modes and genres as semiotic resources are particularly of interest for my
analytical framework.
3.5. Shortcomings of social semiotics and the socio-cultural perspective
It is important to note that the systemic functional framework is an epistemological construct,
similar to the notion of multimodality, which is produced to systematically frame and analyze
certain aspects of reality, while it also has limitations. Although Halliday unequivocally admits the
impossibility to capture cultural and situational contexts in their whole semiotic entirety, he claims
that focusing on the context of situation makes the phenomenon “more describable” (1978: 109).
The multimodal perspective and the overall theoretical perspective of social semiotics has also been
criticized, particularly because of its limitations for giving voice to a multiplicity of voices, and for
considering the complexity of human cognition with relation to sensory information. Multimodal
analyses often tend to be predominated by audio-visual perception rather than exploring the
possible trajectories of human perception (including, but not limited to, touch and smell). By
16
For instance, the design of places and artifacts in the Metrotopia case-study followed particular intertextual discourses
(and visual styles) on superheroes and comic books. On the other hand, the co-designers of PAL explain their initial
purpose as creating a “SL-like” visual style.
68