pdf 820Kb - INSEAD CALT

Evaluation report of the use of Onto-Logging
platform in the user site
Deliverable ID: D8b
Page : 62 of 110
Version: 1.0
Date: 27 january 2004
Status: Final
Confid.: Public
knowledge retrieval dimension aiming at evaluating the capability of the system to support
the more basic knowledge retrieval processes (navigating and searching knowledge); second
a more advanced dimension more centred on the users and groups of people in the
organization, and the advanced means to support these users and groups (via personalization
mechanisms, support for knowledge sharing, etc.).
5.4.1 Evaluating of the basic knowledge retrieval
Questionnaire 2 (Ontologging Project Questionnaire) was used to collect the “user” feedback,
related to the main usage of the Ontology system. In particular, this questionnaire has helped
to collect information related to the use of the central Ontologging tool: the DUI (Distributed
User Interface). It has to be reminder that this tool allows the final user to visualize
knowledge, to navigate into the knowledge, to search knowledge, and also to add new
knowledge items (knowledge capitalization).
Some additional feedbacks were collected from the different focus groups and interviews that
were organized. Finally, experiments (via scenarios) were used to validate different usages
and in particular to identify the difficulties, and to elicitate (cognitive walkthrough) the
internal cognitive process followed by the end users.
5.4.1.1 Evaluating the main tool (DUI)
The main tool (the DUI) was well perceived and considered as adequate, but some
improvement would be well appreciated.
Bellow, are some opinions extracted from the questionnaire:
The system is all right …
“The word enjoy is not correct (but) the DUI is ok and the experience is satisfactory”, “(liked) the
flexibility of the tool to navigate the taxonomy, enabling and disabling the desired relationships,
choosing which concept to see and which one not”; “(liked) to have a global and comprehensive view
of the elements, entities and items involved in the tendering process”, “(liked) the possibility to search
information through link in natural language”, “(liked) using the interface for navigation”.
… but not perfect.
“The way of showing all the information related with one instance is not very useful, is much better
viewing everything navigating the ontology”, “Could be more attractive” , “(problem with)
Navigation when there is too much documents stored”, “Sometimes it’s difficult to know the best way
to perform the searches” , “(disliked) The hierarchical representation of the information. I prefer a
more graphical layout.”
5.4.1.2 Usage scenarios
More interesting are the usage scenarios of the tools that help to understand how the end user
perceived and appropriated the system.