Agile Performance Testing - Testing Experience

and software engineering is not yet like other engineering
fields.
For example, our colleagues in civil engineering build
bridges out of standard components and materials like
rivets and concrete and steel, but software is not assembled
out of standard components to any great extent
yet. Much is still designed and built to purpose, often
from many purpose-built components.
Our colleagues in aeronautical engineering can build
and test scale models of airplanes in wind tunnels.
However, they have Bernoulli’s Law and other physics
formulas that they can use to extrapolate from scale
models to the real world. Software isn’t physical, so physics
doesn’t apply.
There are a select few software components for which
some standard rules apply. For example, binary trees
and normalized relational databases have well-understood
performance characteristics. There’s also a fairly
reliable rule that says once a system resource hits about
80% utilization, it will start to saturate and cause nonlinear
degradation of performance as load increases.
However, for the most part, it is difficult and unreliable
to extrapolate results from smaller environments under scaleddown
loads into larger environments.
Here’s a case study of a failed attempt to extrapolate from small
test environments. One of our clients built a security manage-
www.testingexperience.com
Figure 1: An unrealistically scaled down test environment
ment application that supported large, diverse networks. The
application gathered, integrated, and controlled complex data
related to the accounts and other security settings on servers in a
managed network.
Before they became our client, they ran performance tests in a
The Magazine for Professional Testers
Figure 2: A realistic test environment
67