Steven Pinker -- How the Mind Works - Hampshire High Italian ...

130 I HOW THE MIND WORKSthat makes it impossible to think clearly. Certainly learning plays anenormous role in connectionist modeling. Often a modeler, sent back tothe drawing board by the problems I have mentioned, will take advantageof a hidden-layer network's ability to learn a set of inputs and outputsand generalize them to new, similar ones. By training the livingdaylights out of a generic hidden-layer network, one can sometimes get itto do approximately the right thing. But heroic training regimes cannot,by themselves, be the salvation of connectoplasm. That is not becausethe networks have too little innate structure and too much environmentalinput. It is because raw connectoplasm is so underpowered that networksmust often be built with the worst combination: too much innatestructure combined with too much environmental input.For example, Hinton devised a three-layer network to compute familyrelationships. (He intended it as a demonstration of how networks work,but other connectionists have treated it as a real theory of psychology.)The input layer had units for a name and units for a relationship, such as"Colin" and "mother." The output layer had units for the name of the personso related, such as "Victoria." Since the units and connections arethe innate structure of a network, and only the connection weights arelearned, taken literally the network corresponds to an innate module inthe brain just for spitting out answers to questions about who is relatedto a named person in a given way. It is not a system for reasoning aboutkinship in general, because the knowledge is smeared across the connectionweights linking the question layer to the answer layer, rather thanbeing stored in a database that can be accessed by different retrievalprocesses. So the knowledge is useless if the question is changed slightly,such as asking how two people are related or asking for the names andrelationships in a person's family. In this sense, the model has too muchinnate structure; it is tailored to a specific quiz.After training the model to reproduce the relationships in a small,made-up family, Hinton called attention to its ability to generalize to newpairs of kin. But in the fine print we learn that the network had to betrained on 100 of the 104 possible pairs in order to generalize to theremaining 4. And each of the 100 pairs in the training regime had to befed into the network 1,500 times (150,000 lessons in all)! Obviouslychildren do not learn family relationships in a manner even remotely likethis. The numbers are typical of connectionist networks, because they donot cut to the solution by means of rules but need to have most of theexamples pounded into them and merely interpolate between the exam-