Annual Report 2010 - Fachgruppe Informatik an der RWTH Aachen ...

Metric-based evaluation of software repositories for process assessment
Holger Schackmann
The development of a large portfolio of software projects raises several managerial
challenges, like balancing resource allocation between different projects, and aligning
development processes to the standards of the organization. Hence the project status and
process quality characteristics, like planning precision or problem resolution speed, must be
monitored continuously in order to identify development process weaknesses, and assess
process improvements. Collecting the required data by regularly project status reporting can
be expensive and intrusive, and furthermore ignores the past history of a process. This
motivates mining data that is routinely collected in software repositories like change request
management systems.
The goal of the methods and tools developed in this thesis is to make better use of the data
collected in software repositories for process assessment. The core of the presented solution is
the declarative language ITMS for the specification of metrics on issue tracking systems. This
language facilitates a compact and precise description of metrics on a high abstraction level.
The presented reference implementation of the language can be flexibly adapted to different
software repositories. Moreover the metrics specified in ITMS are easily maintainable, such
that an iterative procedure for development and validation of metrics can be applied.
To ease the systematic interpretation of measurement results, a meta-model for quality models
will be presented. Such a quality model represents the relation between subjective quality
characteristics and the measurements. These concepts are demonstrated in a quality model
editor and evaluation tool. The evaluation tool supports the classification of measurement
results based on empirical comparison data. This facilitates a pragmatic and realistic
interpretation of the measurement results.
Applicability and scalability of the developed methods and tools are demonstrated with case
studies in the context of industrial software development as well as in the context of open
source software.
Evaluators: Prof. Dr. rer. nat. Horst Lichter, RWTH Aachen University
Prof. Dr. rer. nat. Kurt Schneider, Leibniz Universität Hannover
Date of oral exam: 6 May 2009
490