Onto.PT: Towards the Automatic Construction of a Lexical Ontology ...

7.2. Ontologising performance 125
Algorithm
Precision (%)
Nouns (800 tb-triples)
Recall (%) F1(%) F0.5(%) RF1(%)
RP 99.8 82.0 90.0 95.6 99.8
AC 95.4 78.7 86.3 91.5 95.4
NT 96.1 76.1 84.9 91.3 96.1
NT+AC 96.5 75.9 85.0 91.5 96.5
PR 56.3 53.4 54.9 55.7 56.3
MD 52.8 85.2 65.2 57.1 52.8
Verbs (800 tb-triples)
Precision (%) Recall (%) F1(%) F0.5(%) RF1(%)
RP 99.7 84.5 91.5 96.2 99.7
AC 92.3 83.0 87.4 90.3 92.3
NT 95.2 80.1 87.0 91.7 95.2
NT+AC 95.2 80.1 87.0 91.7 95.2
PR 52.0 56.9 54.3 52.9 52.0
MD 69.6 87.1 77.4 72.5 69.6
Adjectives (800 tb-triples)
Precision (%) Recall (%) F1(%) F0.5(%) RF1(%)
RP 100.0 94.8 97.3 98.9 100.0
AC 95.2 93.0 94.1 94.7 95.2
NT 96.1 91.3 93.6 95.1 96.1
NT+AC 96.3 91.3 93.7 95.3 96.3
PR 70.6 73.6 72.1 71.2 70.6
MD 61.6 96.8 75.3 66.5 61.6
Adverbs (476 tb-triples)
Precision (%) Recall (%) F1(%) F0.5(%) RF1(%)
RP 100.0 90.2 94.9 97.9 100.0
AC 99.2 92.5 95.7 97.8 99.2
NT 96.8 91.7 94.2 95.8 96.8
NT+AC 98.3 89.5 93.7 96.4 98.3
PR 92.4 91.7 92.1 92.3 92.4
MD 64.8 95.8 77.2 69.2 64.8
Table 7.4: Results of ontologising samples of antonymy tb-triples of TeP in TeP,
using all TeP’s antonymy relations as a lexical network N.
RF1 always higher than 90%, and F1 always higher than 84%. This confirms our
initial intuition on using these algorithms.
However, it is not expected to extract a complete lexical network from dictionaries,
and even less from other kinds of text. Even though dictionaries have an
extensive list of words, senses and (implicitly) relations, they can hardly cover all
possible tb-triples resulting from a sb-triple, especially for large synsets.
In tables 7.5, 7.6, 7.7 and 7.8 we present the results of a more realistic scenario,
because we only use part of TeP’s lexical network, respectively 50%, 25%
and 12.5%. As expected, performance decreases for smaller lexical networks, but
it decreases more significantly for some algorithms than for others. For instance,
RP’s performance decreases drastically, and it has never the best F1 nor RF1, as it
had when using all the network. On the other hand, RP+AC is the algorithm that
performs better with missing tb-triples. Besides having the best precision in most
scenarios, this algorithm has always the best F1, F0.5 and RF1. There are also some
situations where AC, NT and NT+AC have a very close performance to RP+AC.
The main difference between the previous kind of scenario and a real scenario
is that the part of the lexical network used was selected randomly, which tends to