Lecture Notes in Computer Science 3472

460 Wolfgang Prenninger, Mohammad El-Ramly, and Marc Horstmann
• Kahlouche et al. [KVZ98] describe a special case which does not fit in any
of the previous two. Here, the SUT is stimulated with inputs and the observable
behavior of the SUT is monitored and saved completely during test
execution. Then the monitored trace is translated to the abstract level. Since
in this approach the abstract test case has a tree-like structure, the test is
evaluated to pass iff the monitored trace is a branch of the test case structure.
As stated above, the abstraction/concretization relation is only indicated in the
literature. The main part is hidden in the mechanisms which concretizes the
abstract test cases or abstracts the monitored behavior, respectively. There is no
information about what advantages each of the approaches has, e.g., concerning
scalability or performance.
15.9 Conclusion
In this chapter we reviewed eight case studies from different domains applying
model-based testing mostly in an industrial context. In conclusion, we make four
important observations.
First, application of model-based testing is motivated by the fact that real
world systems (SUTs) are getting more and more complex. Currently, most
systems are tested in an unstructured way by manually written test cases. Due
to the increasing complexity, it is getting more and more difficult to achieve
sufficient test coverage by this approach effectively. Additionally, test suites are
growing huge. For example, in [FKL99], the Architecture Validation Suite (AVS)
developed for PowerPC architecture consisted of about 87,000 test cases for the
32-bit design and 150,000 test cases for the 64-bit design. More general, it is
getting too hard to validate systems against their requirements directly and
effectively.
Second, we observed that all case studies follow a common abstract process.
The process builds an abstract test model of the system’s behavior, which incorporates
only the crucial aspects of system in an abstract way. Due to the
reduction of complexity in the test model, validating the test model is considerably
easier and more effective than directly validating the complex SUT. Then,
by means of test case specification, generation, concretization, execution and
evaluation, the behavior of the SUT is verified against the abstract test model.
This approach leads to a structured test process and enables wide and measurable
test coverage. For building abstract test models, we identified four classes
of abstraction techniques that were applied in the case studies.
Third, we observed that model-based testing starts to be broadly applied in
industry in the domain of processor verification. This stems from the fact that
there is a well-understood development process with well-defined abstraction
levels (e.g., from VHDL down to RTL) in this domain. Actually, this enables
partial automation of the abstract test model building. In the other case studies
the test models are built in an ad-hoc manner due to the lack of well-defined
abstraction levels in their domains. Nevertheless these case studies are promising
proofs of concept.