Onto.PT: Towards the Automatic Construction of a Lexical Ontology ...

8.4. Using Onto.PT 151
In addition to the previous baseline, we had two official runs where Onto.PT
was used to perform an additional expansion on the 67 topics containing verb
phrases (VPs) with only one verb. There, the verbs were disambiguated, and their
synonyms were used as search alternatives. The main idea behind this expansion was
the improvement of the system’s recall. Still, only alternatives with more than 20
occurrences in the the corpora provided by AC/DC were used. The only difference
between these two runs was that, in number 2, disambiguation was performed using
the Bag-of-Words algorithm, while run number 3 used the Personalized PageRank.
Moreover, after the official evaluation, we sent additional unofficial runs, where, besides
other experiments, we had similar runs to 2 and 3, but this time, the category
of all the topics was disambiguated and expanded as well.
In order to illustrate how expansion worked, figure 8.6 presents the expansions
of the category and the VP of the previously shown topics, obtained with the Personalized
PageRank. For the sake of simplicity, we omitted the hypernymy pattern
from the category expansion.
Topic
Original
Category
Expanded Original
VP
Expanded
5 tribo grupo OR tribo habitavam habitar OR
colonizar OR
povoar OR
ocupar
6 viajantes ou viajante OR peregrino OR escreveram redigir OR
exploradores viageiro OR passageiro OR
escrever OR
caminhante OR viandante OR
explorador
grafar
7 sambistas sambador OR sambista abordam tratar OR
apalavrar OR
abordar OR
versar
Figure 8.6: Category and VP expansions in Rapportágico, using Onto.PT.
Given the simplistic approach followed by Rapportágico and the high complexity
of Págico, we can say that the obtained results were interesting. Rapportágico’s
performance was below most of the human participants, but it was better than
RENOIR (Cardoso, 2012), the other automatic participant. Nevertheless, RENOIR
also followed a simplistic approach, and was heavily penalised by the large number
of given answers per topic (100). The most relevant conclusions for our research was
that the runs where VPs were expanded into their synonyms performed better than
the baseline approach. Among these two runs, Personalized PageRank performed
better than the Bag-of-Words method.
The results of the official participation of Rapportágico in Págico are shown in
table 8.7, for each run. In the same table, we present the results of the best human
participation (actually, a groups of participants), ludIT (Veiga et al., 2012), which
show that we are still very far from a human approach to this task, and we show the
results of the best run of RENOIR. Performance is given by the following measures:
• Answered topics: number of topics with at least one given answer.
• Given answers: total number of given answers.