Model-Based Testing - Testing Experience

Let’s assume that a tester performs a review using a flow model.
While reviewing, he might run into the following text. It’s a very
simple example from a functional design, be it a use case or some
other design document:
“After opening the web site the ‘log on’ screen is shown in
which the user can enter his username/password. After validation
the user is shown the ‘choose update product’ screen”.
The point is not that this description says nothing about how
to validate the username/password combination. This may very
well – and for good reason – be written down in some other artifact.
What’s missing here is what should happen if the username/
password combination does not compute. Then what? Back to
the log on screen? Most probably, but how many times? Infinitely,
or is there some security rule that enforces a maximum of three
tries after which a suspend mechanism must be invoked?
One could say that a tester, especially if he’s well-trained and
experienced, should and will find this flaw anyway. That’s true.
However, if a tester, under time pressure, has to read through a
design artifact of dozens of pages, he’s bound to miss some of
the flaws. Testers are not perfect. And even when he picks it up,
it might be very tempting to think (to know!?) that after three
tries, the suspend mechanism must be invoked. So he does not
question the artifact. But that’s interpreting the design. Testers
are human!
Discussing the model
A typical product of a traditional review is a list of comments and
questions. In practice, it’s a list of design flaws and errors. Let’s
be honest, this can be perceived rather negative by the designer.
With model-based reviewing, a shift is made towards the positive.
If a tester can’t put the design into a proper flow, there may be a
couple of reasons. Firstly, the tester may not have the proper background
to do so – he just lacks subject matter knowledge – or it
has to do with the design itself – too little detail, inconsistent, incomplete,
... – or a combination of factors. In any case, something
needs to be done, measures need to be taken.
The best way forward is to discuss the design with all parties
involved. With a traditional review, designers may take remarks
personally, an attack on ‘their work’. Model based reviewing will
take the sting out of this ‘design discussion’: you’re not questioning
the work of someone else, you’re putting your own model up
for questioning! Showing that you created a model out of a design
artifact itself will prove you’ve invested time and effort into
understanding what another designer created.
Suppose a tester wants to discuss the example mentioned before.
The tester can now use the process flow as a communication tool
with the designers of the functional description. The goal is to
complete the model by asking questions like: ‘What happens if
validation fails? ’, or ‘What will happen if validation fails over and
over again?’ In practice, both designers and testers will be adjusting
the model in no time. After the model is finished, the tester
needs to emphasize that all changes to the model imply changes
to the original artifact. It’s up to designers to make these changes.
www.testingexperience.com
Finalizing the model
As stated earlier, model-based reviewing takes place at the stage
‘intake test base’. An important aspect of this stage is a common
understanding of the scope of the testing at hand. Merely agreeing
on testing functional design ‘FU_update_product v0.9.3’ or
use case ‘UC001.C version 1.4’ isn’t enough. To what extent will
this be tested? The finalized model serves as a reference for the
agreements on this. If all paths of the flow model will be tested,
only the test depth needs to be decided, but if the design is too
elaborate to test all paths, including all alternate paths, and every
possible error handling path, then a different way of drawing a
path (different coloring, dotted or not) can be applied to indicate
whether or not that (part of the) path is included in the test at
hand.
Model based test design
Model based reviewing is time consuming, and up to this point
no test case is created and no test is executed. Therefore model-based
test design steps in. Models are unambiguous (at least
good models are). So a test suite derived from such an unambiguous
model consists of a predictable set of test cases, provided by
a formal test design technique. One of the nice features of the
Internet is that it hosts an increasing number of automated test
design tools. With the help of these tools, a test suite can be built
out of models in a matter of minutes, where manual test case
specification would easily take days or more. Thus, the extra effort
put into creating models during test preparation is won back
several times over during the test specification phase!
Advantages of model-based test services
Applying models in our day to day test activities shows several
benefits:
For test preparation:
▪ Creating and reviewing models not only finds defects in the
test base, but also enhances our understanding of the application
under test
▪ It will make the test base more clear and objective.
▪ Using models removes the ‘personal element’ from reviewing.
For test specification:
▪ Accessibility and traceability back to the test base of the test
suite increase.
▪ The width and depth (scope) of the test suite becomes much
clearer, since it is the scope of the model.
▪ Test specification might very well be automated.
It’s plausible that model-based services also provide advantages
for test execution. Increasing the quality of documentation by
applying model-based reviewing has two advantages. First it’s
likely that the quality of the delivered software increases, as the
improved documentation is used by programmers as well. This
could speed up test execution. Furthermore, a major barrier is removed
for test automation as the inconsistencies in documentation
are now gone.
The Magazine for Professional Testers
19