2 Why We Need Model-Based Testing

12 Testing Systems
with Complex State
This chapter discusses various approaches to on-the-fly testing of closed systems.
(We discuss on-the-fly testing of reactive systems in Chapter 16.)
The main difference between on-the-fly testing and offline testing is that in the
case of on-the-fly testing, action sequences are not known beforehand; that is, there
is no pregenerated test suite.
The use of on-the-fly testing is most relevant when the state space of the model
is too large to be covered comprehensively. It may still be possible to use various
finitization techniques discussed in earlier chapters to cope with this problem; onthe-fly
testing is an alternative approach that may be more suitable in some situations,
for example, when existence of a test suite is not relevant, or when it is not clear
how to finitize or abstract the model.
Another case when on-the-fly testing is useful is when implementation under test
(IUT) has hidden states that are not distinguished by the model. In other words, if a
state is revisited in the model, the corresponding state in the implementation may be
new. In such a case a test suite that provides transition coverage of the model might
not expose an error because a transition in the model has to be traversed multiple
times before the IUT is in a state where the error occurs. The error discussed in
Chapter 2 is an example.
The key concept of on-the-fly testing is a strategy. Intuitively, a strategy is a
function that during each step of testing selects which action to apply next. A strategy
may be very cheap; for example, it may make a random choice of what action
to take, or it may be very expensive and try to compute the best possible choice
according to some optimization criterion. A strategy can use additional memory
besides the model state to record prior action selection history, or it can be memoryless
and forget prior choices. Strategies can use coverage points to help exercise
the implementation thoroughly. There is a whole spectrum of possible strategies.
We illustrate how one can use the basic strategy that is supported by the library and
how one can define customized strategies for on-the-fly testing.
more free ebooks download links at:
http://www.ebook-x.com
191