Multimodal Semiotics and Collaborative Design

(Kress, 2010). The analysis of the collaborative processes also shows that the design problems and
the design solutions often develop in a mutual deliberation, even though the project brief is
sketched out from the beginning. This is in line with Cross’ notion that design is some type of a
‘reflexive conversation with the situation” (2007 [1990]). The co-designers in the three case
studies generated particular environments for particular types of interactions, the semiotics of
which essentially depends on the possible set of things that they can mean, and the things that can
be done with them.
Affordances as meaning and action potentials
To summarize the discussion up to here, both the theory and my empirical analysis show that
designing is a sign-making activity, the designed places and artifacts are multimodal sign systems,
and the co-designers are socially situated sign-makers. They shape the meaning potentials by
arranging socially available semiotic resources, and embed their rhetorical intentions into the
designs they co-produce (Kress 2010).
The analyses showed that affordances are related to both making of meaningful associations
(through compositions of modes, genres and representations) and practical functionalities
(through forms, structures and usability). For instance, colors can signify direction of movement,
music and space can be intertextually organized in making of avatar experience, and surfaces of
objects can be used for various purposes, including transparent textures, interactive texts, images,
videos, and interactive Web pages. The over-emphasis on the lack of particular physical constraints
(such as gravity or health) led the co-designers to develop new ideas and affordances specifically
designed for avatar interaction, including flying structures, use of extravagant colors or forms to
emphasize the virtuality of the place.
For sign-makers, affordances are multimodal compositions of various semiotic resources that are
formed by available materials to present certain meaning and action potentials. On the other hand,
for users, affordances can be constructed on different semiotic associations, often resulting in
unprecedented behaviors in different contexts. Therefore, I explained the relationship between
affordances and potentialities by the systemic functional matrix, which emphasizes the role of
experiential, interpersonal and textual meta-functions in making of semiotic texts. In a multimodal
semiotic composition of design elements, signification of intertextual metaphors refers to the
interpersonal meta-function, while the experiential meta-function directs focus to movement and
facilitation of individual and/or social activities. From the sign-user’s perspective 1 , affordances and
constraints are not only signifiers (Norman 2008), they can also represent action potentials in
1
Here, I refer to content generators as sign-users, too, as they are also users of SL’s GUI and the resources it presents.
251