Innovation and Ontologies

Ontology Engineering 213
1.1.2.3 Gómez-Pérez’ Work on Evaluation
One of the more interesting research issues in ontology design
is the formal evaluation of the created ontology.
(Gómez-Pérez, Juristo & Pazos, 1995)
With various contributions since 1995, Asunción Gómez-Pérez explores some general questions
on evaluation, e.g. which elements should be evaluated and by which set of criteria. She also lists
possible types of errors made when domain knowledge is structured in an ontology (Gómez-
Pérez, 1995; Gómez-Pérez, Juristo & Pazos, 1995; Gómez-Pérez, 2001; Gómez-Pérez, 2004).
The following paragraphs give a structured overview of the mentioned aspects.
The general questions guiding an ontology evaluation, introduced in 1995 as ‘primary ideas’, give an
overview on the reasons and contents of ontology evaluation (Gómez-Pérez, 1995). Although far
from being a guideline, Gómez-Pérez’ general questions are essential for any evaluation.
Why?
What?
When?
How?
Where?
Who?
Against what?
Guiding Questions for Ontology Evaluation
To guarantee endusers the correctness and completeness of the ontologies’ definitions,
documentation and software.
• intermediate or final definitions
• set of definitions
• documentation
• software environment
• iterative process
• performed during and between phases of the Lifecycle
• early detection of wrong, incomplete, or missed definitions
standard techniques
anywhere, e.g. at the lab (technical evaluation) or at the end user location (assessment)
• development team (technical properties of the definitions)
• other development teams (technical properties)
• endusers (utility within a given organization or by other software agents)
• ontology evaluation: against a frame of reference (a set of competency questions,
requirements, or the real world)
• software evaluation: against its requirements
table 56 Questions guiding evaluation (based on Gómez-Pérez, 1995)
Objective of the evaluation process is to determine the degree of correctness of the ontology.
Looking at the scope of definitions and axioms, the extent of inference has to be figured out. To
guide evaluation, the following criteria were identified: consistency, completeness, conciseness,
expandability and sensitiveness.
Consistency describes whether valid input data can result in contradictory conclusions:
“A definition is consistent if and only if the individual definition is consistent and no
contradictory sentences can be inferred from other definitions or axioms.” (Gómez-Pérez, 2004).