Multimodal Socialsemiotik & Levende Billeder

The dissertation sketches out possibilities for further development in multimodal
theory (2.4 Nye teoretiske implikationer af multimodalitet, i.e. “new theoretical
implications of multimodality”). Also, the most crucial theoretical implications of
these possible developments as well as way of handling the entailing challenges are
discussed. Among the topics of this chapter are the multimodal understanding of the
communicative situation and a fundamental understanding of multimodal meaning
making based on insights from sensory integration and synaesthesia. The difference
between ambient and demarked semiotic stimuli is discussed in an attempt to
handle the discrepancy between the semiosis of encoding versus the semiosis of
decoding. In addition, the chapter sketches out a fundamental understanding of the
hierarchy of the different senses, semiotic stimuli and modes in the communicative
situation. The chapter also suggests a model which describes a unified multimodal
understanding of the choice of resources in relation to form and content.
Section 2.5 Levende billeder (i.e. “moveing images”) explores both technical and
theoretical aspects of moving images. With emphasis on the possibility for change
over time (i.e. alternation), moving images are defined as dynamic two‐dimensional
visual semiotic stimuli stretched over time. The technological and semiotic
development within the field of moving images is described as an interrelated
phylogenetic progression. The dissertation places multimodal social semiotics in
relation to earlier theories of moving images. Among other things, multimodal social
semiotics has common lineage with film semiotics, and in its current form it is
heavily influenced by cognitive film theory, gestalt theory and neo‐formalism. The
relatively few theoretical investigations of moving images which have been
conducted from a multimodal perspective to date are discussed, and it turns out that
so far no detailed descriptions of the grammatical meaning potential of moving
images exists. Subsequently, the segmentation of moving images is discussed,
resulting in a suggestion of a twofold segmentation following both rank and
temporality.
The description part of the dissertation demonstrates that it is indeed possible to
describe the grammatical meaning potential of moving images. In order to do so, the
dissertation sets off in a critical discussion of previous descriptions of still images in
visual social semiotics and theoretical multimodal reflections on moving images. This
section discusses the detailed descriptions of the grammatical meaning potential of
still images proposed by e.g. Kress & van Leeuwen (2006) and O’Toole (1994) as well
as briefer explorations of moving images by e.g. O’Halloran (2004) and Baldry &
339