Automatic Extraction of Examples for Word Sense Disambiguation

CHAPTER 2. BASIC APPROACHES TO WORD SENSE DISAMBIGUATION 28
Kernel-based are the methods that try to find more general types of relations (and not linear
as just noted above) in the FVs. Their popularity in the past few years has notably increased
which can be seen from their growing participations in recent conferences as Senseval-3 for
example (see Section 4.5). Examples of applications of kernel methods in supervised approaches
are the ones described by Murata et al. (2001); Boser et al. (1992); Lee and Ng (2002); Cristianini
and Shawe-Taylor (2000); Carpuat et al. (2004); Wu et al. (2004); Popescu (2004); Ciaramita and
Johnson (2004).
One of the most popular kernel-methods is the Support Vector Machines (SVM)s presented by
Boser et al. (1992). As Màrquez et al. (2007) report, SVMs are established around the principle of
Structural Risk Minimization from the Statistical Learning Theory (Vapnik, 1998). In their basic
form SVMs are linear classifiers that view input data as two sets of vectors in an n-dimensional
space. They construct a separating hyperplane (in geometry hyperplane is a higher-dimensional
abstraction of the concepts of a line in a n-dimensional space) in that space, which is used to
separate two data sets. To calculate the margin between those data sets, two parallel hyper-
planes are constructed - one on each side of the separating hyperplane, which are directed to the
two data sets. Naturally, a good separation is considered to be achieved by the hyperplane that
has the largest distance to the neighboring data sets. In cases where non-linear classifiers are
desired the selected SVM can be used with a kernel-function.
Discourse properties are considered by the Yarowsky bootstrapping algorithm (Yarowsky,
1995). This algorithm is semi-supervised (see Section 2.4) which makes it hardly comparable
with the other algorithms in that section but it is considered (Màrquez et al., 2007) relatively
important for the following work on bootstrapping for WSD. It uses either automatically or man-
ually annotated training data that is supposed to be complete (to represent each of the senses
in the set) but not necessarily big. This initially smaller set is used together with a supervised
learning algorithm to annotate other examples. If for a given example the annotation is being
accomplished with a higher degree of confidence it is added to the ”seed” set and the process is
further continued.
Similarity-based is a family of methods that are most relevant to our thesis and thus we
provide some more in-depth information about them. However, our aim is still the attempt to
give an overview of those methods so that a better understanding of our use of them can be
accomplished. Approaches of this kind are very often used in supervised WSD because they
carry out the disambiguation process in a very simple way. They classify a new example via
a similarity metric that compares the latter to previously seen examples and assigns a sense
to it - usually this is the MFS in a pool of most similar examples. During the years, probably
because of its increased usage, the approach has gained a wide variety of names: instance-based,
case-based, similarity-based, example-based, memory-based, exemplar-based, analogical. As a
result of the fact that the data is stored in the memory without any restructuring or abstraction