pdf 820Kb - INSEAD CALT

Executive Summary
Evaluation report of the use of Onto-Logging
platform in the user site
Deliverable ID: D8b
Page : 5 of 110
Version: 1.0
Date: 27 january 2004
Status: Final
Confid.: Public
The main goal of this document is to propose a framework for the evaluation, and its
application for the evaluation of the Ontologging System – a next generation ontology-based
knowledge management platform - that has been designed as part of the Ontologging IST
project.
This document is structured in several sections.
The first section presents the methodological aspects and principles of the evaluation, and in
particular describes what are the objective of the evaluation, what are the different phases for
the evaluation (formative or summative), and what are the different categories of methods
(observational, users’ opinion, interpretative, etc.) that can be used. This section also
proposes to use of an adapted version of the Donald Kirkpatrick model, in order to evaluate
the different levels of efficiency and effectiveness achieved by the solution.
The second section presents how these principles are to be applied to the concrete evaluation
of the different components of the Ontologging platform. In particular, this section identifies
the activities that are going to be evaluated (structuring knowledge, capitalizing knowledge,
searching knowledge, and sharing knowledge), selects the different methods to be used and
establishes an action plan.
The third section (chapter 4) is related to the execution of the action plan of the evaluation. It
describes the setting of the evaluation (what is the context of the evaluation at Indra, what is
the profile of the participants of the evaluation). It also presents on the evaluation methods
used and the elaboration of the evaluation instruments, and collection of the data.
Then, the fourth section comprises an analysis of the data, and some preliminary results of
the evaluation.
Finally, the last section draws the main conclusion of this evaluation, and defines some future
work in this domain to be conducted.