Connectionist Modeling of Experience-based Effects in Sentence ...

Chapter 3
Connectionist Modelling of Language
Comprehension
3.1 Structure and Learning
Connectionist networks are prototypical exposure-based models. Or, more precisely,
they are the implementation of non-committed exposure-based accounts. Non-committed
should mean here accounts without any specific assumptions about structural levels or
grain sizes, nor about the linking between corpus regularities and behavior. In literature
there does not seem to be an agreement about what models to call connectionist. It
must be mentioned that there are hybrid models that use parallel distributed activation
spreading between symbolic entities on the one hand (e.g. Just and Carpenter, 1992;
Lewis and Vasishth, 2005), and there are connectionist models that use hand-designed
architectures and local representations on the other hand (e.g. Dell et al., 2002; McClelland
and Elman, 1984; Rohde, 2002). I am concerned here only with “fully connectionist
models” using fully distributed representations and no pre-designed internal structuring.
The most important feature that distinguishes a connectionist network model of that
kind from symbolic models is its architecturally constrained highly adaptive learning
ability. Connectionist models are functional problem solving machines that, depending
on the specific learning algorithm and certain architectural properties, are able to find
the optimal solution to any task representable as input-output pairs. The design of
symbolic models mostly involves many assumptions about the desired processes which
are hard-coded into the system. For example, it has to be specified how to categorize
and represent the input. A connectionist system on the other hand starts from zero
without any presumptions. The structure of the internal input representation is shaped
during the learning process depending on the task requirements. Obviously the information
about the structure that linguists annotate to word strings of natural language
is already there in the plain strings. Extracting the underlying structure of the input
requires information about sequential and temporal relations between input chunks. For
that reason time is an important component of cognitive tasks. In particular language
transports highly-structured information while being entirely sequential. A memory of
earlier input and the representation of temporal relations between input chunks pro-
47