2 Why We Need Model-Based Testing

148 Testing Closed Systems
B()
A()
0 1
C()
D()
Figure 8.6. Transition coverage versus path coverage.
2
The number of messages sent by each test case is a consequence of the traversal
algorithm used by the offline test generator tool, otg. The postman tour algorithm
achieves full transition coverage: every transition in the FSM is taken at least once.
The postman tour is minimal: the total number of transitions in the test suite is the
minumum needed to achieve transition coverage; the algorithm avoids traversing
the same transition more than once, if that is possible. This explains the shapes of
the test suites displayed in Figures 8.1 and 8.2. (In the second test suite, it is pure
luck that the longer message happens to be sent first.)
These examples confirm that minimal transition coverage is not sufficient to
expose some defects. Consider the simple example in Figure 8.6. Minimal transition
coverage will generate two test cases: A(),B(),C() and A(),D(). The assumption
is that the model state really does determine the behavior; there is no need to
cover every path; D() will not be affected by B() preceding it. This assumption
can be false if the implementation has hidden state that is not represented in the
model. In that case, we should also attempt some amount of path coverage by also
testing the paths A(),B(),D(), and maybe even A(),B(),B(),D(). In our client/server
implementation, the hidden state is the client’s receive buffer.
These are limitations of minimal transition coverage, not offline test generation
in general. We can imagine an offline test generator with a more thorough traversal
algorithm. But the requirement in offline testing that the entire test suite must be
computed in advance discourages thorough coverage. Moreover, it is not feasible
to explore the true FSM of many “infinite” model programs at all. And, when the
implementation exhibits nondeterminism, it is wasteful to compute paths that may
never be taken. For these reasons, the tools emphasize on-the-fly testing, where the
tests are computed as the test executes. In contrast with the limited choice of traversal
algorithms built into the otg tool (just one at this writing), the ct tool provides rich
opportunities for testers to program their own on-the-fly testing strategies, as we
shall see in Chapters 12 and 16.
8.6 Exercises
1. Write a stepper for the three newsreader implementations you wrote for the
problems in Chapter 5. Generate a test suite from the newsreader model program
more free ebooks download links at:
http://www.ebook-x.com