Connectionist Modeling of Experience-based Effects in Sentence ...

More documents

Recommendations

Info

4.4 Forgetting Effects Results for 3a Similar to the experiment in Vasishth et al. (2008) the assessed regions in the simulation were the three verbs V3, V2, V1 and the post-V1 region. The V2 region contains is no datapoint in the ungrammatical condition because the verb is dropped in the testing stimuli. Figure 4.6 shows GPE values for the SRNs trained and tested on the simplified English grammar without commas. The left panel shows both conditions after one training epoch. On the right results after three full corpus runs are shown. The ungrammatical condition is called drop-V2 and the grammatical is indicated by nodrop. The pattern was exactly as expected. The SRNs predicted a drop-V2 advantage on V1 and post-V1. No effect was predicted on V3 because no difference in stimuli and probability between the conditions is present at this point. GPE 0.0 0.2 0.4 0.6 0.8 1.0 English without commas (epoch 2) V3 V2 V1 post-V1 Region drop-V2 no-drop GPE 0.0 0.2 0.4 0.6 0.8 1.0 English without commas (epoch 3) V3 V2 V1 post-V1 Region drop-V2 no-drop Figure 4.6: Simulation 3a: English doubly-embedded ORC. The graphic shows the GPE value on the three verbs and the subsequent region of the grammatical (nodrop) and ungrammatical (drop-V2) condition. The left panel shows GPE after two epochs of training, the right panel shows GPE after 3 epochs. 3b: English with commas The commas serve as clause boundary markers. They appear in English SRCs subsequent to nouns only. In the ORC, on the other hand, commas appear after nouns in the beginning of the sentence and after the verbs in the end. In a double-embedded ORC there would be a comma after V3 and V2. Thus, the grammatical/ungrammatical sequence pair is not NNNVVV vs. NNNVV anymore but rather N,N,NV,V,V vs. N,N,NV,V. The comma is a category with only one token which attaches to nouns or 75
Chapter 4 Two SRN Prediction Studies verbs and is not involved in long-distant dependencies. Hence, the activation pattern representing it should not be too complex. In fact the learning of comma usage in ORCs can be scaled down to a counting recursion problem of the pattern aabb instead of abba. As discussed in chapter 3 counting recursion is the easiest of the three recursion types for both humans and connectionist networks (Christiansen and Chater, 1999). Thus, it is very likely that the inclusion of commas facilitates processing in the grammatical condition lowering the respective GPE values. (25) English with commas: a. SRC: S1 , V2 O2 , V3 O3 , V1 O1 b. ORC: S1 , S2 , S3 V3 , V2 , V1 O1 (26) Example test sentences: a. the banker , that the banker , that the senators phone , understands , attacks the reporters . (no-drop) b. the lawyer , that the senator , that the judges attack , praises the judge . (drop-V2) Results for 3b See figure 4.7 for the results of simulation 3b after one (left panel) and three epochs (right panel). Compared to simulation 3a there was a global improvement for both conditions. The most dramatic improvement happened on V3, which is predicted almost without errors after three epochs. Looking at the first epoch there was more improvement due to comma insertion on V1 for the grammatical condition. In result the V1 error was the same in both conditions. However, after subsequent training the no-drop condition did not change on V1 whereas the drop-V2 condition improved further resulting in a drop-V2 preference on V1. The opposite happened on post-V1 where training had affected the no-drop condition more. Here training did not affect the ungrammatical condition at all. In summary, there was a comma insertion × condition × training interaction, resulting in a drop-V2 preference after completed training. The stable error on post-V1 in the drop-V2 condition can be interpreted as a floor effect. The prediction of the determiner and the noun is very good already with a GPE value around 0.1. It is very unlikely that the SRN learns the perfectly correct probabilities resulting in a GPE value of zero even after many epochs. Therefore, on the post-V1 region improvement by training is only possible for the slightly worse grammatical condition, which is why the two conditions settle on the same error value after three epochs. In conclusion, the insertion of commas definitely helps to make better predictions. However, training effects seem to be driven by rather local consistency, affecting the ungrammatical condition more than the grammatical. Thus, looking at V1 after three epochs the drop-V2 preference seems to be stable for English center-embedding. 76
Page 1 and 2:
Connectionist Modeling of Experienc
Page 3 and 4:
Acknowledgments I am grateful to Be
Page 5 and 6:
Contents 3.3.4 Summary . . . . . .
Page 7 and 8:
List of Tables 2.1 Languages with s
Page 9 and 10:
Chapter 1 Preliminaries and the ACT
Page 11 and 12:
Chapter 1 Preliminaries (1) a. The
Page 13 and 14:
1.3 Psycholinguistic Aspects Chapte
Page 15 and 16:
Chapter 1 Preliminaries referrentia
Page 17 and 18:
Chapter 1 Preliminaries empirically
Page 19 and 20:
Chapter 1 Preliminaries rial, in Le
Page 21 and 22:
Chapter 1 Preliminaries are limited
Page 23 and 24:
Chapter 1 Preliminaries predictor.
Page 25 and 26:
Chapter 2 Issues in Relative Clause
Page 27 and 28:
Page 29 and 30:
Page 31 and 32: Chapter 2 Issues in Relative Clause
Page 45 and 46: clear predictions for head-final RC
Page 51 and 52: Reading time [ms] 400 600 800 1000
Page 55 and 56: OUTPUT PUT PLAN Chapter 3 Connectio
Page 57 and 58: Chapter 3 Connectionist Modelling o
Page 63 and 64: German Word Order Chapter 3 Connect
Page 65 and 66: Length-Adjusted Reading Time (ms) C
Page 69 and 70: Chapter 4 Two SRN Prediction Studie
Page 71 and 72: 4.1.3 Training and Testing Chapter
Page 73 and 74: GPE 0.0 0.2 0.4 0.6 0.8 1.0 English
Page 75 and 76: (23) [V1 [N1 V2 de ORC ] N2 de SRC
Page 81: Chapter 4 Two SRN Prediction Studie
Page 85 and 86: (27) German with commas: a. SRC: S1
Page 87 and 88: 4.4.4 Discussion Chapter 4 Two SRN
Page 91 and 92: Bibliography M. H. Christiansen. Th
Page 93 and 94: Bibliography E. Gibson and J. Thoma
Page 95 and 96: Bibliography J. W. King and M. Kuta
Page 97 and 98: Bibliography F. Reali and M. H. Chr
Page 99 and 100: Appendix A Statistics SRC ORC regio
Page 101 and 102: Appendix B Grammars B.1 English (wr
Page 103 and 104: B.2 German {numREL, Nnom, RC RCpure
Page 105 and 106: B.3 Mandarin Rel : SRC (0.85) | ORC
show all

Connectionist Modeling of Experience-based Effects in Sentence ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?