A Wordnet from the Ground Up

4.4. Benefits of Extracted Patterns 123Hum. eval. Ranking Prec. plWN Rel. R Inst.[%] 70% 60% 50% [%] Inst.ESP- 39 8 22 43 36 501 1.0 3982ESP-nm 47 5 14 62 37 561 1.54 6435ESPmorf- 45 13 18 71 39 361 0.75 2600ESPfree- 43 9 12 23 29 567 1.36 4621EST- 54 10 27 – 30 651 1.71 4917EST-nm 59 42 90 – 35 571 1.7 4457EST+nm 37 18 32 52 27 1312 2.38 10000Table 4.2: The influence of the extended reliability measure and changes in the pattern form (“Hum.eval.” – precision based on human judgement, “Ranking” – the number of the top instancesabove the precision threshold, “Prec. plWN” – precision in relation to plWordNet, “Rel. R”– relative recall relative to ESP-)manual evaluation. It means that EST-nm starting from the same seeds acquired fromplWordNet goes beyond the source and extracts many instances which are not describedin plWordNet. This is a very promising feature concerning the potential application inexpanding plWordNet.We also observed that the value of the original reliability measure (4.1) decreasesvery fast. After the sixth iteration it goes far below 10-12. This explains the dropof the number of newly extracted instances. Applying the modified reliability formula(4.6) circumvents the problem.Another matter of concern is the scheme of the patterns adjusted for Polish. It isclear that the application of the adjusted patterns produces better precision EST- andEST-nm in comparison to ESP- and ESP-nm. In the case of EST+nm, utilising thegeneric patterns, the precision is lower, but its relative recall shows its potential inextracting new instances. At the cost of reduced precision, the number of extractedinstances increases by the factor 2.38 (the total number of the extracted instancesdepends on the number of instances above the threshold).The second group of experiments was performed only for Estratto using genericpatterns and the extended reliability measure, i.e. for EST+nm. The aim was todetermine the influence of the algorithm parameters on the result. The followingdependencies were investigated:1. the influence of the confidence threshold on the precision of instances achievedwithin subsequent iterations,2. the influence of the number of seeds on the induced patterns, and then theinfluence of the relation between instances and patterns induced by them,3. the influence of the number of the top k patterns selected for the next iterationon the stability of the algorithm and the precision of instances,