pdf 820Kb - INSEAD CALT

1 Introduction
Evaluation report of the use of Onto-Logging
platform in the user site
Deliverable ID: D8b
Page : 6 of 110
Version: 1.0
Date: 27 january 2004
Status: Final
Confid.: Public
The main goal of this document is to propose and to apply a framework for the evaluation of
the Ontologging System – a next generation ontology-based knowledge management
platform– that has been designed as part of the Ontologging IST project.
This document tries to provide a complete view of the evaluation:
The rational for the evaluation, what are the methods available, what methods have been
selected for evaluating the Ontologging system, how these methods are operationalized,
executed, and how is the result analysed.
This document is structured in several sections.
The first section presents the methodological aspects and principles of the evaluation: what is
the evaluation for? What can be evaluated? What should be evaluated? What are the different
methods available to conduce an evaluation? Etc.
The second section presents how these principles are to be applied to the concrete evaluation
of the different components of the Ontologging platform (a next generation, ontology-based
knowledge management platform). In particular, this section establishes an action plan for
evaluating the system, and in particular defines the methods that have been chosen to capture
the data.
The next section (chapter 4) is related to the execution of the action plan of the evaluation. It
presents concretely the setting of the evaluation (what is the context of the evaluation, who
are the participant), the elaboration of the instrument and the collection of the data.
Then, the following sectio n is concerned with the analysis of the data, and of the result of the
evaluation (this section may be empty at an initial stage of this document). This section also
presents how the different results are taken into account in order to improve the system.
Finally, the last section draws the main conclusion of this evaluation, and defines some future
work in this domain to be conducted.