02.05.2014 Views

Proceedings - Österreichische Gesellschaft für Artificial Intelligence

Proceedings - Österreichische Gesellschaft für Artificial Intelligence

Proceedings - Österreichische Gesellschaft für Artificial Intelligence

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

Scalable precision<br />

Rank \ Size 2 3 4 5 6 7 8 9 10 11 12 all<br />

1-500 1.00/1.00 - 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 0.95/0.95 1.00-1.00<br />

10001-10500 1.00/1.00 0.80/0.80 0.99/0.99 0.96/0.97 0.99/1.00 0.87/0.92 0.98/0.99 0.96/0.97 0.99/0.99 1.00/1.00 0.98/0.98 0.98-0.99<br />

20001-20500 0.96/0.98 0.92/0.87 0.97/0.99 0.96/0.96 0.96/0.98 0.92/0.92 0.98/0.98 0.93/0.96 0.97/0.99 0.93/0.95 0.93/0.96 0.96-0.98<br />

30001-30500 0.97/0.98 0.84/0.86 0.98/0.98 0.91/0.95 0.92/0.98 0.88/0.92 0.88/0.98 0.89/0.95 0.97/0.99 - 0.83/0.96 0.97-0.98<br />

40001-40500 0.83/0.92 0.85/0.85 0.83/0.96 0.77/0.87 0.88/0.97 0.75/0.86 0.89/0.97 0.91/0.94 0.85/0.99 0.91/0.95 0.25/0.95 0.83-0.95<br />

50001-50500 0.16/0.55 0.15/0.49 0.16/0.93 0.40/0.85 0.47/0.96 0.33/0.84 0.38/0.97 0.44/0.93 - - - 0.17-0.82<br />

Strict precision, thresh = (0 * size) / 12<br />

Rank \ Size 2 3 4 5 6 7 8 9 10 11 12 all<br />

1-500 1.00/1.00 - 0.99/0.99 1.00/1.00 0.99/0.99 1.00/1.00 1.00/1.00 1.00/1.00 0.95/0.95 1.00/1.00 0.43/0.43 0.98-0.98<br />

10001-10500 1.00/1.00 0.40/0.40 0.97/0.99 0.82/0.88 0.97/0.98 0.42/0.65 0.93/0.96 0.71/0.81 0.95/0.95 1.00/1.00 0.85/0.76 0.92-0.95<br />

20001-20500 0.95/0.98 0.75/0.62 0.92/0.96 0.78/0.82 0.85/0.94 0.56/0.61 0.88/0.93 0.43/0.66 0.80/0.93 0.68/0.76 0.50/0.68 0.84-0.91<br />

30001-30500 0.96/0.97 0.53/0.57 0.95/0.96 0.65/0.77 0.67/0.92 0.33/0.57 0.75/0.93 0.50/0.65 0.67/0.93 - 0.00/0.67 0.90-0.91<br />

40001-40500 0.79/0.90 0.68/0.65 0.66/0.92 0.33/0.58 0.57/0.87 0.23/0.45 0.45/0.88 0.46/0.60 0.25/0.92 0.00/0.71 0.00/0.65 0.62-0.85<br />

50001-50500 0.14/0.54 0.12/0.38 0.07/0.88 0.40/0.57 0.33/0.86 0.00/0.43 0.00/0.87 0.00/0.59 - - - 0.13-0.73<br />

Strict precision, thresh = (1 * size) / 12<br />

Rank \ Size 2 3 4 5 6 7 8 9 10 11 12 all<br />

1-500 1.00/1.00 - 0.99/0.99 1.00/1.00 0.99/0.99 1.00/1.00 1.00/1.00 1.00/1.00 0.95/0.95 1.00/1.00 1.00/1.00 0.99-0.99<br />

10001-10500 1.00/1.00 0.40/0.40 0.97/0.99 0.82/0.88 0.97/0.98 0.42/0.65 0.93/0.96 0.71/0.81 0.95/0.95 1.00/1.00 0.96/0.97 0.93-0.96<br />

20001-20500 0.95/0.98 0.75/0.62 0.92/0.96 0.78/0.82 0.85/0.94 0.56/0.61 0.88/0.93 0.43/0.66 0.80/0.93 0.68/0.76 0.79/0.91 0.84-0.92<br />

30001-30500 0.96/0.97 0.53/0.57 0.95/0.96 0.65/0.77 0.67/0.92 0.33/0.57 0.75/0.93 0.50/0.65 0.67/0.93 - 0.00/0.90 0.90-0.92<br />

40001-40500 0.79/0.90 0.68/0.65 0.66/0.92 0.33/0.58 0.57/0.87 0.23/0.45 0.45/0.88 0.46/0.60 0.25/0.92 0.00/0.71 0.00/0.88 0.62-0.86<br />

50001-50500 0.14/0.54 0.12/0.38 0.07/0.88 0.40/0.57 0.33/0.86 0.00/0.43 0.00/0.87 0.00/0.59 - - - 0.13-0.74<br />

Strict precision, thresh = (2 * size) / 12<br />

Rank \ Size 2 3 4 5 6 7 8 9 10 11 12 all<br />

1-500 1.00/1.00 - 0.99/0.99 1.00/1.00 0.99/0.99 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00-1.00<br />

10001-10500 1.00/1.00 0.40/0.40 0.97/0.99 0.82/0.88 0.99/0.99 0.67/0.80 0.96/0.98 0.93/0.95 0.99/0.99 1.00/1.00 1.00/1.00 0.96-0.98<br />

20001-20500 0.95/0.98 0.75/0.62 0.92/0.96 0.78/0.82 0.93/0.97 0.89/0.84 0.95/0.97 0.93/0.94 0.92/0.98 0.88/0.91 0.93/0.98 0.92-0.96<br />

30001-30500 0.96/0.97 0.53/0.57 0.95/0.96 0.65/0.77 0.86/0.96 0.83/0.84 0.75/0.96 0.50/0.92 1.00/0.98 - 1.00/0.98 0.92-0.95<br />

40001-40500 0.79/0.90 0.68/0.65 0.66/0.92 0.33/0.58 0.78/0.94 0.68/0.79 0.73/0.94 0.85/0.90 0.25/0.97 1.00/0.92 0.00/0.96 0.69-0.90<br />

50001-50500 0.14/0.54 0.12/0.38 0.07/0.88 0.40/0.57 0.33/0.93 0.00/0.75 0.00/0.93 0.00/0.88 - - - 0.13-0.77<br />

Strict precision, thresh = (3 * size) / 12<br />

Rank \ Size 2 3 4 5 6 7 8 9 10 11 12 all<br />

1-500 1.00/1.00 - 0.99/0.99 1.00/1.00 0.99/0.99 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00-1.00<br />

10001-10500 1.00/1.00 0.40/0.40 0.99/0.99 1.00/1.00 0.99/0.99 0.67/0.80 0.96/0.98 1.00/1.00 0.99/0.99 1.00/1.00 1.00/1.00 0.97-0.98<br />

20001-20500 0.95/0.98 0.75/0.62 0.97/0.98 1.00/1.00 0.93/0.97 0.89/0.84 0.98/0.98 1.00/1.00 0.96/0.99 0.94/0.96 1.00/1.00 0.95-0.97<br />

30001-30500 0.96/0.97 0.53/0.57 0.97/0.98 0.94/0.98 0.86/0.96 0.83/0.84 0.75/0.97 1.00/1.00 1.00/0.99 - 1.00/1.00 0.94-0.97<br />

40001-40500 0.79/0.90 0.68/0.65 0.79/0.95 0.79/0.90 0.78/0.94 0.68/0.79 0.95/0.97 0.85/0.96 1.00/0.99 1.00/0.96 0.00/0.98 0.78-0.93<br />

50001-50500 0.14/0.54 0.12/0.38 0.07/0.91 0.40/0.87 0.33/0.93 0.00/0.75 0.50/0.97 0.00/0.94 - - - 0.14-0.80<br />

Strict precision, thresh = (4 * size) / 12<br />

Rank \ Size 2 3 4 5 6 7 8 9 10 11 12 all<br />

1-500 1.00/1.00 - 0.99/0.99 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 1.00-1.00<br />

10001-10500 1.00/1.00 1.00/1.00 0.99/0.99 1.00/1.00 1.00/1.00 1.00/1.00 0.96/0.98 1.00/1.00 1.00/1.00 1.00/1.00 1.00/1.00 0.99-0.99<br />

20001-20500 0.95/0.98 1.00/1.00 0.97/0.98 1.00/1.00 0.98/0.99 1.00/1.00 0.98/0.98 1.00/1.00 1.00/1.00 0.97/0.98 1.00/1.00 0.98-0.99<br />

30001-30500 0.96/0.97 1.00/1.00 0.97/0.98 0.94/0.98 1.00/0.99 1.00/1.00 0.75/0.97 1.00/1.00 1.00/1.00 - 1.00/1.00 0.97-0.98<br />

40001-40500 0.79/0.90 0.93/0.95 0.79/0.95 0.79/0.90 0.96/0.99 0.77/0.92 0.95/0.97 1.00/1.00 1.00/1.00 1.00/0.98 0.00/0.98 0.84-0.96<br />

50001-50500 0.14/0.54 0.13/0.53 0.07/0.91 0.40/0.87 0.33/0.98 0.00/0.88 0.50/0.97 0.00/0.98 - - - 0.14-0.82<br />

Table 2: Precision depending on size, rank and evaluation criteria.<br />

Rank / Size 2 3 4 5 6 7 8 9 10 11 12<br />

0-9999 934-9% 45-0% 1928-19% 218-2% 1764-18% 485-5% 1340-13% 699-7% 977-10% 1031-10% 579-6%<br />

10000-19999 682-7% 100-1% 1662-17% 307-3% 1649-16% 447-4% 1947-19% 629-6% 1299-13% 611-6% 667-7%<br />

20000-29999 1169-12% 243-2% 2969-30% 699-7% 2416-24% 666-7% 748-7% 289-3% 360-4% 265-3% 176-2%<br />

30000-39999 2223-22% 787-8% 3082-31% 948-9% 1160-12% 571-6% 489-5% 303-3% 260-3% 71-1% 106-1%<br />

40000-49999 4191-42% 1545-15% 1690-17% 689-7% 636-6% 375-4% 369-4% 189-2% 194-2% 45-0% 77-1%<br />

Total 15288-27% 3830-7% 11449-20% 2879-5% 7650-13% 2557-4% 4906-9% 2117-4% 3097-5% 2025-4% 1608-3%<br />

Table 3: Number and distribution over size of the translation hypotheses generated.<br />

478<br />

<strong>Proceedings</strong> of KONVENS 2012 (LexSem 2012 workshop), Vienna, September 21, 2012

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!