Multimodal Semiotics and Collaborative Design

Xavier: It was very, EXTREMELY colorful. Pink, red, shiny blue whatever, very strong colors
that you would never use in RL, I would never make something like that. But for something
like PAL, and in SL, it really worked out well.
Similar to color, sound is used a differentiating mode in PAL’s design. As mentioned before, the
primary functional purpose of the sim is to present music to visitors in a social environment, while
the four streaming locations (plus various other social areas) differ in style of content. The visual
style of the sim signifies the visual appropriation of its content, and its affordances for creation of
experiential and interpersonal meaning potentials.
Shaggy: So I had to think about what music is, and how to reflect music visually because SL is
mostly about visuals. And here the idea was to make a good visual about music. And to me
music is rhythm. And so, lining cubes was a good idea to make a visual rhythm. So that’s
basically why I came up with this idea in the first place.
The visualization of music and rhythm shaped the design of experiential features, while visual
rhythm’s interpersonal potentials were considered as the identifying feature of the new design.
PAL’s overall layout functions as a textual element in its overall semiotic cohesion; thus represents
certain modal affordances. While such relations refer to the relations between music and visual
rhythm in PAL’s design, they also frame and shape the experiential and interpersonal relations
between the avatars and the virtual place.
The workshop projects also show similar characteristics in terms of the use of layout and color as
modes for semiotic expression. I have already illustrated some of the multimodal arrangements in
the Venus and the Four Moons group’s red lipstick-shaped spaceship and the Awesome Three
group’s pink stereo and dance floor designs. In the first example, the group used the color red, and
the shape of a lipstick purposefully as a symbolic referents to feminism. In addition, their avatars
were used as signifiers to specific concepts and visually coded with associated colors (i.e. Isis-ice -
blue). In the second example, students’ goal was dissemination of messages through text, image,
color and sound. Their choice of color in their main object stereo (or variety of colors in the dance
floor) aimed to support the semiotic coherence of their message. The aim of the students was not
only to capture the attention by colors, but also to support the positive social message of the design.
To summarize this section, my observations showed that the co-designers often utilize semiotic
potentials of various modalities (i.e. text, image, music, video, color, layout, visual rhythm,
interactivity) to emphasize particular affordances and constraints of multimodal arrangements,
and to achieve coherence between their rhetorical intentions and the designed forms. In table 8.2.,
I summarize the findings about the use of modes and modal compositions in the three case-studies.
231