file - ChaSen - 奈良先端科学技術大学院大学

Figure 4.1. Annotation Tool for Answer Articles.The annotators can start the annotation at any article in the dataset andcan change the result of the annotation at any time. The datasets with myexperiments are composed of answer articles and corresponding question articles,but the question articles were not presented to the annotators. When annotatorsput an annotation on an article, the tool logged the time stamp at that point intime.The annotators could interrupt and resume their annotation in my experiments.They annotated the articles independently in different places and alldiscussion of judging articles was prohibited. The annotators were also not permittedto annotate answer articles by referring to the question articles.Before annotating the articles, the rater was given examples of the results ofannotating some typical answer articles (see Appendix B).54