pdf 820Kb - INSEAD CALT

Evaluation report of the use of Onto-Logging
platform in the user site
Deliverable ID: D8b
3.3.2.1 Level 0: Do the system performs technically well?
Page : 28 of 110
Version: 1.0
Date: 27 january 2004
Status: Final
Confid.: Public
This level addresses the evaluation of the technical characteristic of the Ontologging platform
& components. In particular, if the different tools (DUI, Ontology editor, User profile editor,
the reconciliatory agent, etc.) are easy to set-up, are reliable, are fast, are interoperable, etc.
3.3.2.2 Level 1: Users’ acceptance (do they like it)?
This level is related to the ergonomic of the Ontologging component such as the perceived
speed, the aesthetics, the readability of the fonts, the navigability, and the interactivity (drag
& drop capabilities, etc.).
3.3.2.3 Level 2: Assessment: Do the approach/system function?
This level addresses the functionality of the different components of Ontologging and for
instance: can we create any ontology or what are the limitations? What are the functions
currently implemented related to knowledge searching? What are the different means for an
agent to intervene to support the collaborative authoring process? What can be evaluated, etc.
3.3.2.4 Level 3: Transfer: Is the approach/system used?
This level addresses the use of the Ontologging systems in the tester companies (and in
particular Indra) for supporting their knowledge management processes. This level also
assesses the condition of use of Ontologging, in particular related to other knowledge
management approaches (such as the Meta4 product Knownet). For instance, it evaluates the
use of the system in better supporting the work processes (for instance the tender process)
and the collaborative dimension (for instance, are the agent mechanisms effectively used to
support collaborative ontology authoring?). Other questions: are the advanced searching
capabilities (inference) used? How often the different Ontologies are updated? Do user use
knowledge evaluation features?
3.3.2.5 Level 4: Impact: Measures the impact on the organization
This level addresses the effectiveness of the Ontologging system in making the employee to
better manage the knowledge of the company and making it operate better (for instance
improving the relationships with its customers, increasing its flexibility and its reactiveness).
More concretely, it may provide some assessment in how ontologies contribute to provide a
better management of the knowledge (because it enables flexibility and also the support for
the structuring of the knowledge-oriented activities). Finally, this level will also try to assess
the cognitive transformation of the user, and the changes in his work practices: are the beliefs
and the behaviours of the users different been radically transformed by the system (or has the
user radically learned)?