Sommerville-Software-Engineering-10ed

724 Chapter 24 ■ Quality management
24.5.3 Measurement ambiguity
When you collect quantitative data about software and software processes, you have
to analyze that data to understand its meaning. It is easy to misinterpret data and to
make incorrect inferences. You cannot simply look at the data on its own. You must
also consider the context in which the data is collected.
To illustrate how collected data can be interpreted in different ways, consider the
following scenario, which is concerned with the number of change requests made by
a system’s users:
A manager decides to measure the number of change requests submitted by customers
based on an assumption that there is a relationship between these change
requests and product usability and suitability. She assumes that the higher the
number of change requests, the less the software meets the needs of the customer.
Handling change requests and changing the software are expensive. The organization
therefore decides to modify its process with the aim of improving
customer satisfaction and, at the same time, reducing the costs of making
changes. The intent is that the process changes will result in better products and
fewer change requests. Processes are changed to increase customer involvement
in the software design process. Beta testing of all products is introduced, and
customer-requested modifications are incorporated in the delivered product.
After the process changes have been made, the measurement of change
requests continues. New versions of products, developed with the modified
process, are delivered. In some cases, the number of change requests is
reduced; in others, it is increased. The manager is baffled and finds it impossible
to understand the effects of the process changes on the product quality.
To understand why this kind of ambiguity can occur, you have to understand why
users might make change requests:
1. The software is not good enough and does not do what customers want it to do.
They therefore request changes to deliver the functionality they require.
2. Alternatively, the software may be very good, and so it is widely and heavily
used. Change requests may be generated because many software users creatively
think of new things that could be done with the software.
Increasing the customer’s involvement in the process may reduce the number of
change requests for products where the customers were unhappy. The process
changes have been effective and have made the software more usable and suitable.
Alternatively, however, the process changes may not have worked, and customers
may have decided to look for an alternative system. The number of change requests
might decrease because the product has lost market share to a rival product and there
are consequently fewer product users.