Connectionist Modeling of Experience-based Effects in Sentence ...

Chapter 4 Two SRN Prediction Studies
verbs and is not involved in long-distant dependencies. Hence, the activation pattern
representing it should not be too complex. In fact the learning of comma usage in ORCs
can be scaled down to a counting recursion problem of the pattern aabb instead of abba.
As discussed in chapter 3 counting recursion is the easiest of the three recursion types
for both humans and connectionist networks (Christiansen and Chater, 1999). Thus,
it is very likely that the inclusion of commas facilitates processing in the grammatical
condition lowering the respective GPE values.
(25) English with commas:
a. SRC: S1 , V2 O2 , V3 O3 , V1 O1
b. ORC: S1 , S2 , S3 V3 , V2 , V1 O1
(26) Example test sentences:
a. the banker , that the banker , that the senators phone , understands , attacks
the reporters . (no-drop)
b. the lawyer , that the senator , that the judges attack , praises the judge .
(drop-V2)
Results for 3b
See figure 4.7 for the results of simulation 3b after one (left panel) and three epochs (right
panel). Compared to simulation 3a there was a global improvement for both conditions.
The most dramatic improvement happened on V3, which is predicted almost without
errors after three epochs. Looking at the first epoch there was more improvement due
to comma insertion on V1 for the grammatical condition. In result the V1 error was the
same in both conditions. However, after subsequent training the no-drop condition did
not change on V1 whereas the drop-V2 condition improved further resulting in a drop-V2
preference on V1. The opposite happened on post-V1 where training had affected the
no-drop condition more. Here training did not affect the ungrammatical condition at all.
In summary, there was a comma insertion × condition × training interaction, resulting
in a drop-V2 preference after completed training. The stable error on post-V1 in the
drop-V2 condition can be interpreted as a floor effect. The prediction of the determiner
and the noun is very good already with a GPE value around 0.1. It is very unlikely
that the SRN learns the perfectly correct probabilities resulting in a GPE value of zero
even after many epochs. Therefore, on the post-V1 region improvement by training
is only possible for the slightly worse grammatical condition, which is why the two
conditions settle on the same error value after three epochs. In conclusion, the insertion
of commas definitely helps to make better predictions. However, training effects seem to
be driven by rather local consistency, affecting the ungrammatical condition more than
the grammatical. Thus, looking at V1 after three epochs the drop-V2 preference seems
to be stable for English center-embedding.
76