A Technical History of the SEI

Developing a Measurement System That Supports an
Organization’s Goals
The Challenge: Software Project Measurements That Support Business
Goals
Despite significant improvements in implementing measurement programs for software development,
a large percentage of measurement programs are not successful. Organizations often do not
achieve the potential benefits of a sound measurement program due to the inconsistent construction
and interpretation of indicators derived from measurement data. One of the dangers in enterprises
as complex as software development and support is that there are potentially so many
things to measure that users are easily overwhelmed by the opportunities. The search for the
“right” measures can easily become confusing when the selection is not driven by the information
requirements to be addressed by the measures. A successful measurement program is more than
collecting data. The benefits and value of doing software measurement comes from the decisions
and actions taken in response to analysis of the data, not from the collection of the data. The SEI
was challenged to develop a measurement and analysis methodology to support the goals of an
organization and to ensure that data is not collected for the sake of collection alone.
A Solution: Goal-Driven Software Measurement—Goal-Question-Indicator
To address the challenge, the goal-question-metric (GQM) methodology, introduced and described
by Basili and Rombach 24 [Basili 1988, Rombach 1989], was enhanced and augmented by
the SEI into the goal-question-(indicator)-metric methodology (GQIM), a disciplined approach to
defining a set of measures and indictors related to the goal. The goal-driven software measurement
process produces measures that provide insights into important management issues as identified
by the business goals. Since the measurements are traceable back to the business goals, the
data collection activities stay better focused on their intended objectives. In goal-driven measurement,
the primary question is not “What metrics should I use?” but “What do I want to know or
learn?” [Rombach 1989].
The steps of the approach are organized into three sets of activities: identifying goals, defining indicators
and the data needed to produce them, and creating an action plan to guide the implementation.
Business goals are translated into measurement goals [Basili 1984, Briand 1996] by refining
them into concrete, operational statements with a measurement focus. This refinement process
involves probing and expanding each high-level goal to derive questions. The questions provide
concrete examples that can lead to statements that identify what type of information is needed.
From these questions, displays or indicators are postulated that provide answers and help link the
measurement data that will be collected to the measurement goals. The goal-driven approach requires
that indicators (charts, tables, or other types of displays and reports) be sketched out and
approved by the intended user. These indicators serve as a requirements specification for the data
that must be gathered, the processing and analysis that must take place, and the schedule for these
24 Basili, Victor R. “Using Measurement for Quality Control and Process Improvement.” 2nd Annual
SEPG Workshop. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA., June
21-22, 1989. No longer available.
CMU/SEI-2016-SR-027 | SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY 152
Distribution Statement A: Approved for Public Release; Distribution is Unlimited.