Automatic Extraction of Examples for Word Sense Disambiguation

CHAPTER 2. BASIC APPROACHES TO WORD SENSE DISAMBIGUATION 27
Decision lists (Yarowsky, 1995; Martínez et al., 2002) as their name suggests are simple or-
dered lists of rules that are of the form (condition, class, weight). Such rules are usually more
easy to understand if thought of as if-then-else rules: if the condition is satisfied then the accord-
ing class is assigned. However, in the form of the rules that we provided above there is also a
third parameter that is taken into account - weight. Weights are used to determine the order of
the rules in the list. Higher weights position the rules higher in the list and respectively lower
weights mean that the rules can be found further down in the ordered list of rules. The order
in the decision lists is important during classification since the rules are tested sequentially and
the first rule that ”succeeds” is used to assign the sense to the example. Usually the default rule
in a list is the last one that accepts all remaining cases.
Decision trees (Mooney, 1996) are basically very similar to decision lists however not this often
used for word sense disambiguation. They as well use classification rules but this time the rules
are not ordered in a list but as an n-ary branching tree structure that represents the training
set. Hence every branch of the tree represents some rule that is used to test the conjunction of
features and to provide a prediction of the class label encoded in the terminal node (also called
leaf node is a node in a tree data structure that has no child nodes). Some of the problems with
decision trees, which makes them not really handy for WSD, are their computational cost and
the data fragmentation (breaking up the data into many pieces that are not close together) that
they employ. The latter leads to immense increase in computation if larger feature spaces are
used. The same result is also triggered by the use of a large number of examples, however, if
fewer training instances are provided a relative decrease in the reliability of the predictions for
the class label can be observed.
Rule combination for supervised word sense disambiguation means that a set of homoge-
neous classification rules is combined and learned only by a single algorithm.
AdaBoost (Schapire, 2003) is a very often used rule combination algorithm. It combines mul-
tiple classification rules into a single classifier. The power of AdaBoost is based on the fact that
the classification rules must not necessarily be very accurate but after they are combined the
resulting classifier has an arbitrarily low error rate.
Linear Classifier or also called binary classifier achieved in the last few decades considerably
low results and thus the highest interest to them was in the field of Information Retrieval.
Those kind of classifiers decide on the classification label based on the linear combination of the
features in their FVs. They aim to group the instances with the most similar feature values.
The limited amount of work on linear classifiers has resulted in several articles as for example
(Mooney, 1996; Escudero et al., 2000b; Bartlett et al., 2004; Abdi et al., 1996; Cohen and Singer,
1999). In case a non-linear problem has to be decided, for which the expressivity of the linear
classifiers is not enough, suggestions for the use of kernel functions (kernel methods) have been
made.