TPT Tutorial - PikeTec

TPT Tutorial 

Version 3.0 

11. March 2008

Contents 

Page 2 TPT Tutorial 

1 Introduction _____________________________________________________________________________ 3 

1.1 What is TPT? _____________________________________________________________________________ 3 

1.2 What is TPT not? _________________________________________________________________________ 4 

1.3 Systematic Testing – What is the Point of it? ____________________________________________ 5 

1.4 Determination of the Test Goals _________________________________________________________ 6 

1.5 Continuous Behaviour ___________________________________________________________________ 7 

1.6 Automated Test Cases for Continuous Behaviour _______________________________________ 7 

1.7 Interfaces of Test Cases __________________________________________________________________ 8 

2 TPT — an introductory Example _______________________________________________________ 10 

2.1 Example “Light Automatics” _____________________________________________________________ 10 

2.2 Executable Test Case Modelling ________________________________________________________ 13 

2.2.1 MATLAB/Simulink Settings _____________________________________________________________ 13 

2.2.2 The interfaces of the test cases ________________________________________________________ 18 

2.2.3 The first and simplest executable testlet _______________________________________________ 19 

2.2.4 More complex test case: “lights on if dark” ____________________________________________ 25 

2.2.5 The second test case: “lights on if light” _______________________________________________ 28 

2.3 Execution of the Modelled Test Cases __________________________________________________ 31 

2.3.1 Test set configuration __________________________________________________________________ 31 

2.3.2 Execution Configuration _______________________________________________________________ 31 

2.4 Assessment _____________________________________________________________________________ 33 

2.5 Report generation_______________________________________________________________________ 34 

2.6 Executable platform lights control test _________________________________________________ 34

TPT Tutorial Page 3 

1 Introduction 

The introduction explains important and basic concepts for the manual. It includes 

answers to questions such as “What is TPT, where can it be used and what can’t it do?”, 

“What is a systematic test?”, “What does continuous behaviour mean?”, “What is a 

reactive test?” and a lot more. 

This introduction will be of particular interest to readers who are not yet familiar with 

testing strategies in general, continuous behaviour of systems or the basic idea of TPT. 

However, this chapter may also offer some new insights to other readers. 

1.1 What is TPT? 

TPT is a method and tool environment for systematic black box testing of continuous 

behaviour 1 . The use of TPT makes sense whenever a system comprised of continuously 

changing states, behaviour or conditions is to be tested (cf. chapter1.5). The special 

features of TPT are in particular: 

• TPT can be used independent of the test computing platform upon which the live 

tests should eventually run and of the language the system to be tested has been 

implemented. Still, TPT supports the fully-automated test execution. (Naturally, 

some technical adjustments of the interfaces or extensions to the infrastructure may 

be necessary for this). TPT offers the possibility of running modelled test cases on 

different platforms. 

• TPT supports systematic testing. This means that during the modelling of test cases 

with TPT – starting from the functionality of the system to be tested – all test cases are 

always viewed together, in order to define an ideal set of test cases which contains few 

or no redundancies and takes into account all relevant situations and test cases. 

The underlying methodology of TPT thereby guarantees that even with a large 

number of relevant test cases clarity will not be lost. The advantage of the systematic 

approach is explained in more detail in chapter1.3. 

1 Please note that in this context “continuous behavior” is time segmented continuous behavior.


The tool environment of TPT is divided into two areas: the graphic user interface (UI) and 

the runtime environment (RE). The UI serves the purpose of modelling test cases by the 

tester and is a “100% pure Java”-implementation, which has currently only been tested in 

Windows. The RE is necessary for the test execution and is implemented in ANSI-C, Java, 

C++, MATLAB/Simulink, LabVIEW or similar, depending on the platform. The reference 

implementation of the RE consists of an abstract machine, implemented in ANSI-C, which 

can calculate highly optimized byte code. 

A clear separation of UI and RE ensures the above-mentioned platform independence of 

the essential core of TPT. 

1.2 What is TPT not? 

Figure 1: Graphical user interface of TPT 

In TPT the selection and modelling of the test cases is performed manually by the tester, 

while the test execution takes place automatically. The belief that TPT could test a 

system in a fully automated manner without any manual effort is thus illusory. 

TPT is not a debugger. This means that while TPT does serve the purpose of uncovering 

errors, it does not facilitate locating the root cause. Thus, after the system behaviour has 

been detected as erroneous for a test case, other tools must be used in order to detect 

the actual location of the error in the system design. 

TPT is a pure black box testing procedure and therefore does not support any 

monitoring of internal data and control flows of the system. This particularly means that 

coverage measurements (as in white box testing with statement coverage, branch 

coverage, path coverage etc.) are not supported. In order to be able to carry out such 

coverage measures other tools must thus be applied in parallel to TPT, which perform 

such structured test analysis.


1.3 Systematic Testing – What is the Point of it? 

This chapter begins by explaining why systematic testing is essential for high-quality 

product development. This section is not a prerequisite for understanding the following 

chapters, they may be skipped. 

Test execution of a system with selected test cases is allows analysis of the system 

behaviour for these selected cases. To uncover the errors of a complex system it is not 

sufficient to view just one test case. On the other hand it is not possible to view all cases. 

Test case design is a trade-off of effort spent in test design against the likelihood of 

finding errors or risk of not finding errors in the system design, i.e. the goal is to reduce 

the risk of undetected errors in the system design getting past the test process. 

Thus, testing is always a random sample procedure in which a subset of relevant test 

cases is selected from the total set of all possible cases. The aim of test case selection is 

to detect as many errors as possible. 

The aim of the test execution is to uncover as many error areas as possible by using as 

few test cases as possible. Thus, each test case that does not uncover any errors is, from 

the test’s point of view, just as disadvantageous as two test cases detecting the same 

error. The selection of these test cases is far from trivial and the most demanding task 

during testing. 

To select the set of test cases, the following approaches are possible but not exhaustive: 

Name Description Advantages Disadvantages 

Ad-hoc 

test cases 

Random 

test cases 

Statistical 

tests 

Evolutionary 

tests 

Systematic 

tests 

The set of test cases is 

selected without any 

defined rules and 

depends on the 

expertise of the tester. 

The selection of test 

cases is automated with 

the help of a random 

generator. 

The test cases are 

randomly generated in 

accordance with a 

statistical distribution 

(e.g. operational profile). 


determined with the 

help of evolutionary 

search procedures. 


selected in accordance 

with specified criteria (e. 

g. coverage measures in 

white box testing or 

“consideration of all test 

cases of the 

This approach does not 

require any planning 

and can therefore be 

useful in very early 

phases of a 

development. 

Manual test case 

selection is not 

necessary. 



necessary; reliability 

statements are 

(theoretically) possible. 



necessary. 

It is possible to take 

specific account of 

criteria with which the 

tester and/or the 

domain expert believes 

that the system will, 

with some probability, 

Ad-hoc test cases are comparatively 

unproductive due to the undefined 

procedure. Most of the time, the error 

detection rate is very low. 

Random tests are still very complex in 

practice, as the amount of test cases 

needed for error detection is usually 

extremely high. Thus, random tests are 

rarely used. 

A statistical distribution function is 

needed. 

A measure is needed for the 

automatic evaluation of test data so 

that interesting test data can be 

detected (example: If a system is 

supposed to control the temperature 

of a cooling circuit of a nuclear power 

plant the measure may be the 

temperature itself. Evolutionary 

optimization is used in order to try to 

find test data which leads to the 

violation of the acceptable 

temperature.) 

The definition of the test cases must 

take place explicitly (to ensure 

controllability of the criteria) and 

mostly manually, which clearly 

increases the effort for the test case 

selection.


environment” in black 

box testing). 

show erroneous 

behaviour. 

Figure 2: Overview of approaches for test case determination 

Although systematic testing requires a significant degree of effort during test case 

selection, it is still very efficient. The reason for this lies in ’high effort’ provided by 

automated test execution (i.e. the actual “testing” of the system) and the ‘lower effort’ 

manual test evaluation of the system behaviour. 

In a complex system many outputs need to be monitored and evaluated, which generally 

cannot be completely automated. This means that the behaviour for each test case must 

be analyzed individually and at least partly manually. Thus, efficient testing needs to 

uncover as many errors as possible with as few test cases as possible. Although ad-hoc 

tests do fulfil the first part they detect only a fraction of the errors in the system. Random 

tests and statistical tests only uncover a large amount of errors once the number of test 

cases is extremely high. Evolutionary tests try to use optimization to generate a relatively 

small number of test cases, which are optimized with regard to a certain measure. This 

procedure is applicable if such an optimization measure can be defined, which, in 

complex systems, may not be possible. 

In systematic testing, the set of relevant test cases is kept small due manual determination 

of prioritised test cases. Here, the tester can consciously choose where to put the 

emphasis regarding selected test cases, and thus differentiate critical aspects from less 

critical ones in the test. Despite the manual effort involved in selecting test cases, 

systematic testing is essential for high-quality system design analysis. 

1.4 Determination of the Test Goals 

Whenever one is testing a system, one has certain goals. In many cases they are not 

explicit– but exist nevertheless. One very natural test goal is to test the functionality of 

the system. This is called a functionality test or logic test. In this case, the test is used to 

check whether the functional requirements of the system have been fulfilled. Accordingly, 

the selected test cases are strongly linked to the system requirements. 

Another test goal, for example, is the real-time test. Here, real-time systems are checked 

to see whether the specified time conditions are met. In these tests, preference of 

definition is given to those test cases in which time critical factors are highlighted. This 

usually means that completely different test cases are needed than for logic tests. 

Therefore it is important to be clear about the test goals before starting to model the test 

cases. A system containing an integrated set of test cases may be partly evaluated 

regarding several test goals at the same time. It is important that the test goal(s) are 

clearly defined to determine which test cases should be examined and which should be 

classified as non-relevant for the test. 

Possible test goals include: 

• Functionality test/Logic test: Test regarding functional requirements from the 

customer specification document 

• Real-time test: Test regarding critical time behaviour for real-time systems 

• Marginal test: Test of behaviour for limit values of data areas of interface data 

• Memory test: Test regarding violations via illegitimate memory accesses 

• Robustness test: Test with data which lies outwith valid input 

• Comfort test: Test of the system’s usability from the user’s point of view 

• Safety test: Test of safety features


• etc. 

Furthermore, in larger systems we focus on a narrower set of goals make the whole more 

manageable (for example “real-time test during the system initialization”). In smaller 

systems – as in the light automatics from chapter 2 – this restriction is not necessary due 

to the trivial functionality. 

The test which is defined in this way – described by the system to be tested, the test goals 

as well as the examined sub-function – is called the test object. 

1.5 Continuous Behaviour 

As mentioned before, TPT supports the testing of continuous behaviour. The simplest 

definition of continuous behaviour is: “Continuous behaviour is the behaviour of a system 

in a dense (real-) time section”. This means: To analyse the behaviour of a system it is 

necessary to continuously watch the input and output interfaces over a certain period of 

time (and not only at one point in time or one sequence of moments), this is called 

continuous behaviour 2 . 

Systems with continuous behaviour are always systems with a high degree of complexity, 

infinitely internal state space, which is why it is not enough to just view the interface 

behaviour at single points in time. From a practical point of view, one can test the 

continuous behaviour for systems whose behaviour “may naturally be described and 

analyzed by means of discrete or continuous signal forms”. Each system that operates with 

signal forms therefore shows continuous behaviour, which can be tested by using TPT. 

This characteristic mainly applies to embedded systems, as these systems always 

communicate in real time with their environment and influence real values of their 

environment. TPT is particularly suitable for embedded systems, although the application 

range – as suggested above – is decidedly broader. 

1.6 Automated Test Cases for Continuous Behaviour 

Each test case requires an executable test driver. The test driver’s behaviour is determined 

by the test case at runtime. This means that the test case specifies how the input of the 

system has to be stimulated. The test driver handles the technical implementation of the 

test case’s intention. 

Often, there is not a specific test driver available for each test case. Instead, there are test 

drivers with a set of controlling parameter values, which allow them to implement many 

different test cases. The controlling parameter information for the test case is in the form 

of parameter sets, files, scripts, data tables or similar. 

For the passive or open loop test, in which the system is being stimulated with certain 

predefined inputs (s1, s2, s3), it is sufficient to implement a test driver which uses fixed 

data tables, thereby stimulating the system. 

TPT is also able to model reactive or closed loop tests. For those tests the test case has 

the ability to react to the system’s behaviour during runtime. (cf. Figure 3 b)). 

2 Please note that real continuous behavior is not possible in sampled systems. Thus the “continuous behavior” is 

only quasi-continuous behavior.


Stimulat ion 

s1 

s2 

s3 

Syst em 

Test driver/ 

Test case 

Stimulat ion 

s1 

s2 

s3 

Syst em 

Test driver/ 

Test case 

a) Open loop test b) Closed loop test 

Figure 3: open loop and closed loop test 

In closed loop tests, the system to be tested and the test driver interact in the loop with 

the help of so-called channels. Channels are continuous variables, which can continuously 

change their value over time. The test case stimulates the input channels of the system 

with the corresponding signal forms and observes the output channels of the system in 

order to react to the system behaviour. 

In open loop tests, the feedback between system and test case does not apply. Thus, the 

test case can only contain predefined fixed signal forms, which must not dynamically react 

to the system behaviour. Therefore a reaction to certain events or states of the system is 

not possible. In this respect, it is usually easier to model open loop tests, but they cannot 

address an important class of test cases, in particular with regard to the reusability of test 

cases. 

1.7 Interfaces of Test Cases 

The channels to be stimulated or to be observed correspond to the input and the output 

channels of the system and thus directly depend on the system and its technical and 

logical interfaces. If a test case runs fully automatically, all input channels of the system 

must be stimulated by the test case so that no input remains “undefined”. Since the 

number of output channels of the system is limited due to technical reasons, each test 

case can, as a basic principle, only observe output channels. This means that each test 

case for a given system must always define all stimulation channels and be able to 

monitor all observation channels. All in all, this means: 

All test cases defined for one and the same system have the same interfaces in the form 

of observation and stimulation channels. 

Stimulation- 

Channels 

s1 

s2 

s3 

Test driver/ 

Test case 

o1 

o2 

Figure 4: Interfaces of a test case 

Observation- 

Channels 

This observation is actually quite trivial. However it is also exceptionally important, since it 

makes clear that different test cases have something in common for one system. As we 

o1 

o2 

Observation


will see later, the basic philosophy of TPT is closely related to the identification of a 

common base of the test cases.


2 TPT — an introductory Example 

TPT addresses a multitude of different facets of testing such as test case specification, a 

formal test description language, a framework for test execution, a modelling language 

for the test result assessments, automated test analysis and much more. In order to 

introduce all these aspects, TPT shall be illustrated below using a simple example. Files 

used in this example may be found in the folder “\examples”. ATTENTION: When 

executing the test cases defined in file "lights_control.tpt", all test results (data, report...) 

will be written to the directory, this file has been opened from. This behaviour may cause 

errors, in case TPT has been installed into a directory it cannot write to. To avoid those 

problems, copy this directory to a writable location on your file-system and open the file 

from before you execute tests. 

2.1 Example “Light Automatics” 

The following example is not a simple “Hello World” example, as it is often used for the 

introduction of a programming language! In order to illustrate the basic ideas of TPT as 

well as the usage of the method and the tool, we will walk through a simple example. 

The system to be tested shall be a simplification of automatic lighting control for the 

headlight of a vehicle with the functionality as described below.


Functional specification of the lights control: 

The lights control operates the headlights depending on the setting of the light 

switch and the light conditions (measured by a sensor): 

[#1] The headlights (headlights) can be switched on or off; there are no 

intermediate settings. 

[#2] The light intensity (illumination) is measured by a light sensor that gives 

the brightness range from 0 – 100% with 0% being absolute darkness and 

100% being the brightest level of light measureable by the sensor. 

[#3] The light switch (switch) has three settings: OFF, ON and AUTO 

(automatic mode). 

[#4] In ON or OFF the headlights shall always be switched on or off accordingly. 

[#5] On AUTO, the status of the headlights depends on the sensed ambient 

light conditions as follows (cf. requirements [#6] to [#9]): 

[#6] If the system starts in AUTO or has been switched to AUTO the headlights 

shall go on immediately if the brightness is 70% or below. 

[#7] If the system starts in AUTO or has been switched to AUTO the headlights 

shall go off immediately if the brightness is above 70%. 

[#8] On AUTO, if the brightness falls below 60% for at least 2 seconds, the 

lights go on. 

[#9] On AUTO, if the brightness exceeds 70% for at least 3 seconds, the lights 

go off again (threshold value). 

The light control system is implemented in MATLAB/Simulink. It consists of a Simulink 

subsystem that has two inputs (light switch setting switch and ambient light intensity 

illumination) and one output (headlights) (cf. Figure 5: Interfaces of lights control 

). 

light switch 

OFF 

AUTO 

ON 

light sensor 

switch 

illumination 

headlights 

light control 

Figure 5: Interfaces of lights control 

headlights 

The implementation of the subsystem “lights control” could for instance be as displayed 

in Figure 6: Implementation of the light control 

Remember that this model is only meant to enhance comprehension of the principles in 

the manual. Since TPT is a black box test procedure, looking at the internal structure of 

the system implementation is neither relevant for test modelling nor for test execution.


Figure 6: Implementation of the light control 3 

For automated testing of a system such as this it is necessary for a test case to define how 

the inputs of the system to be tested shall be stimulated. Conversely, it will be shown that 

in many cases it might be necessary for the test case to be able to react to the behaviour 

of the system. This creates a loop between the system that is to be tested and the test 

case as depicted in Figure 7. 

Stimulation 

illumination 

switch 

Test object 

„lights control“ 

Test case 

headlights 

Figure 7: Loop between system and test case 

With the aid of TPT, test cases are now defined that show actual processes for the “Test 

case” box in Figure 7. 

A decision must be made what general test goal shall be pursued in the test i.e. what is 

the focus of the system testing before this test case definition can begin. 

Relevant test cases differ depending on the test goal: if the test is to examine the 

robustness of the system, test cases are primarily selected that test the system behaviour 

under faulty conditions. However, if the desired test goal is e.g. the comfort test, test 

3 simplified model of implementation 

Observation


cases are primarily relevant that test the usability of the system. The relevant test cases 

turn out very differently depending on the test goal. 

The purpose of this tutorial is to show the logic test of the light control in which testing of 

the specified functional system behaviour is of prime interest. 

A large number of test cases usually have to be modelled to test a test object. One can 

approach this modelling by specifying all abstracted test cases that are considered 

relevant and then implementing them in the tool, thus making the test cases executable. 

This procedure is known as the top-down procedure. One can also begin with a few 

precisely formulated test cases and systematically expand them with other relevant test 

cases. This approach is known as the bottom-up procedure. Which procedure gets chosen 

usually only depends on the fact if an initial set of obvious test cases has already been 

established or if the test is begun afresh. 

2.2 Executable Test Case Modelling 

In the following chapter demonstrates how test cases are modelled and transferred into 

an executable form by using TPT. All steps necessary for the generation and execution of 

a very simple test example are given. 

2.2.1 MATLAB/Simulink Settings 

Alternatively for MATLAB/Simulink the inputs and outputs can be imported. Also the 

MATLAB/Simulink communication has to be established and the test frame needs to be 

generated. 

Start with a new and plain TPT. Choose Execution->Platform in order to set up 

MATLAB/Simulink as test execution environment. Now press Add in the lower left corner 

and select MATLAB/Simulink as platform type (cf. Figure 8) 

Figure 8: Test platform setup 

Figure 8: Test platform setup 

The platform needs some more configurations in the Platform Configuration window 

under the name specified before. Choose the MATLAB version and select Configure (cf. 

Figure 9).


Figure 9: Platform configuration window 

In the configuration window MATLAB/Simulink has to be registered as com server in 

order to enable the communication between MATLAB and TPT (cf. Figure 10). 

Alternatively the communication between TPT and MATLAB can be set up using the RMI 

protocol. It is possible to start MATLAB from here. 

Figure 10: MATLAB configuration


Figure 10: MATLAB configuration 

Next the model file and the Simulink-block of the system under test are to be selected. 

Select the original model file in the Platform Configuration window (cf. Figure 11). 

Figure 11: Platform configuration; model settings 

When the communication with MATLAB is set up and the block for the system under test 

is specified, the interface (namely the inputs and outputs of the selected block) can be 

imported by selecting “Extract interface”. In the lights control example light_switch 

and light_intensity are outputs of TPT but inputs of the system under test. 

Headlight is the input to TPT and can be used for the assessment of the test results (cf. 

Figure 12).


Figure 12: Import test interface 

When the interface between TPT and the Simulink is specified the test-frame can be 

extracted. This can be done automatically when a new test frame model file is specified 

(cf. Figure 13). The new test frame model file then consists of a copy of the selected 

Simulink block and an S-Function block where the execution VM of TPT is located. 

Figure 13: Generate test frame 

By clicking on Generate test-frame the new model file will be generated. When the layout 

is not optimal it may be adjusted manually (cf. Figure 14).


Figure 14: Test frame in MATLAB/Simulink 

Please note: The communication between TPT and MATLAB may be observed in the 

Console view of the Platform configuration window (cf. Figure 15). 

Figure 15: MATLAB-TPT communication console


In order to make the test executable once the test cases have been modelled, the test 

execution configuration is necessary. Please specify here the test frame model file that has 

been generated previously (cf. Figure 16). 

2.2.2 The interfaces of the test cases 

Figure 16: Test execution configuration 

In order to model an executable test case, the interfaces to the system under test must be 

defined. TPT provides a “declaration editor” to enable the specification (name, type, etc.) 

of the required channels (cf. Figure 17). In the example of the lights control the input and 

output channels depicted in Figure 7: Loop between system and test case 

must be declared.


Figure 17: Declaration of the input and output channels 

Following the declaration of the channels, the interfaces (signature) of the test cases must 

be defined. The channels must be selected from a list as observation (input to the test) or 

stimulation (output to the test) channels. 

Figure 17: Signature-definition for test cases 

In Figure 18 the interface is graphically depicted in the signature editor. The headlight 

channel is an observation channel and thus an “input” for the test case. (The test case is 

represented by the box). The two other channels light_switch and 

light_intensity are the corresponding stimulation channels, which are, from the test 

case’s point of view, “exits”. 

This modelling of the interface sets all the prerequisites for the modelling of the first test 

case. 

2.2.3 The first and simplest executable testlet 

The simplest executable testlet consists of a simple automaton with an initial and final 

junction and one state. The state needs to specify the inputs to the system under test. 

To add a testlet to a test case, follow the following steps: 

1. Left-click the new state icon in the Tool and menu browser 

2. Move your cursor into the editor window and left-click to place a new state 

The new state appears both in the editor-window and Testlet browser


3. Right-click on the new state in the testlet browser and rename it to “Simplest Testlet”: 

In order to add an initial and final junction, do the following steps: 

1. Left-click into the “main” in the testlet browser for a better overview of your workspace. 

Then left-click the new junction icon in the Tool and menu browser: 

2. Move the cursor left to the “Simplest State” in the editor window and left-click to place 

the initial junction.


3. Now left-click the new final icon in the tool and menu browser: 

4. Move the cursor right to the “Simplest State” in the editor window and left-click to 

place the final junction: 

In order to draw the transitions from the initial junction to the testlet and to the final 

junction follow the following steps: 

1: Left-click a new transition icon in the testlet browser and move the cursor over the 

initial junction:


2. Left-click and move your cursor over the testlet. You should see a transition pointing 

from initial junction to “Simplest Testlet”: 

3. Now left-click again to complete the connection: 

Follow the same procedure to connect the testlet with the final junction. The test case 

should now look like this:


Figure 18: Simplest executable test 

To configure the behaviour of your testlet, do the following steps: 

1. Left-click the “Simple Testlet” in the testlet browser and left-click again the direct 

definition icon in the editor window: 

2. Now to assign the signature of the testlet move the cursor right to the testlet. 

Notice than the area under your cursor changes the colour to gray. 

3. Left-click this area to choose both the “light_intensity” and “light_switch” as 

output-channels for the “Simplest Testlet”. Now the signature of this state has two 

outputs light_intensity and light_switch.


Figure 19: Outputs of the state 

The content of the simple state needs to be specified as direct definition because it has 

no sub-states and cannot refer to another state. 

Figure 20: Direct definition for light switch on 

We specify the default scenario and set the light switch to one. In this simple test we 

expect the headlights to be on. 

Please navigate back to the main window in the testlet browser. 

In order to make the automaton executable a transition condition for the transition 

between the state and the final junction must be specified. Double click on the transition 

and edit the formal transition condition. In this example the test terminates after five 

seconds.


Figure 21: Transition editor 

Now select the default scenario in the scenario browser. The automaton appears in grey. 

Please select one of the transitions. Since there are no variations modelled in the 

automaton the whole path through the automaton appears in solid black. 

The simple test case can now be executed. See Section 2.3. 

2.2.4 More complex test case: “lights on if dark” 

For the modelling of the test case with TPT it is necessary to first establish the steps the 

test case consists of: 

1 Modelling of the behaviour of the light switch position: 

1.1 At the beginning of the test case turn light switch to ”ON”, ... 

1.2 ... then wait ... 

1.3 ... until 10 seconds have passed, ... 

1.4 ... then maintain switch position and ... 

1.5 ... wait again 4 ... 

1.6 ... until another 10 seconds have passed, ... 

1.7 ... then end of test. 

2 Modelling of the behaviour of the ambient light intensity: 

2.1 From the beginning to the end of the test case – i.e. simultaneously to 1 – 

”light intensity = dark” is simulated until the test case has reached step 1.7. 

Figure 22: Informal process description of the test case 

The first part (“Modelling the behaviour of the light switch position”) describes the state 

of the light switch position. The second part deals with the behaviour of the ambient light 

intensity. Both parts are independent, i.e. there is no direct relation of the light switch 

position to the ambient light intensity and vice versa. These two parts are thus described 

independently. For the entire test case the two parts are run (quasi) simultaneously: While 

4 

The fact that in part 1 of the above description we are waiting twice might not make too much sense at this 

point in time. However, this peculiarity will be explained further down in this chapter.


the light switch position is defined the ambient light intensity is determined 

simultaneously. 

These two parts of the model suffice to specify the test case completely, since all the 

relevant stimuli of the “lights control” system are described (the inputs “light switch 

position” and “ambient light intensity”). At this point the informal description of the test 

case is implemented in the modelling language of TPT. 

The modelling of test cases takes place in TPT by means of special automatons. The 

following example requires a fundamental understanding of automaton notations with 

states, transitions etc. The subsequent explanation of the example does not go into the 

details of the language semantics, but gives only a rough idea of how test cases are 

implemented. For a detailed description of the modelling language please refer to the 

TPT user manual. 

The procedure of the “headlight on and dark” test case is modelled in TPT as depicted in 

Figure 24. 

Figure 23: Process modelling of a test case (in the right-hand window) 

Comments on the modelling of the test case from Figure 24: 

Comment 1: The automaton language used in TPT primarily consists of the “usual” 

components for automaton languages (states, transitions, junctions etc.). A feature 

specific to TPT in comparison with other automaton languages is that the states and 

transitions in TPT are annotated with natural-language, informal texts. The formal 

definitions, which are naturally necessary for the automatic test execution, are hidden 

behind these states and transitions and are not depicted in the graphic. 

The advantage of the informal description is that test cases can also be read by nontesters 

who need not be familiar with the formal modelling language of TPT. 

The reason for the red text is explained later in this chapter. 

Comment 2: The syntactical descriptions behind the natural-language annotations at 

transitions and states can be accessed by double-clicking on the respective element in the 

TPT tool. For instance, the following window opens by double-clicking the first transition 

“switch light condition” (after 10s):


Figure 24: light switch condition transition 

This window depicts the formal condition of the transition (“formal condition”). As we 

can see, the “light switch condition” (after 10 s) transition has the formal condition t >= 

10 s., i.e. after 10 s the transition condition becomes true and the automaton switches to 

the next state. In addition, so-called actions (“actions”) can be defined for the transitions. 

In the case of the above transitions no actions are defined at the moment the transition 

switch takes place. 

Comment 3: Similarly, double-clicking on a state shows the details of the modelling of the 

state. Such a state can be defined either by a sub-automaton (in the sense of hierarchical 

automatons) or by equations. In the sub-automaton-case, the sub-automaton is shown 

after double-clicking. States on the lowest level contain simple equation definitions of the 

modelled stimulation channels. 

Please note that each state – although it is only informally annotated by means of its 

natural-language name in the graphic depiction of the automatons – establishes a 

detailed, precise and clear process description. 

Comment 4: In Figure 24 the overall process of the test case is divided into two: The upper 

part models the procedure for the switch setting, the lower part describes the procedure 

of the ambient light intensity. The horizontal line between the two areas indicates that 

there are two parallel automatons, which run quasi-simultaneously. 

Comment 5: The process of implementing a test case always starts with an initial node, 

which is represented by a solid circle (“ “), and which is only a transitions source. As 

the test case consists of two parallel automatons there are also two initial nodes – one for 

the upper and one for the lower automaton - in which the test case starts simultaneously 

at the time t=0. This means that the initial nodes serve as entry points into the test case 

and determine which transition should be used to change to the first state of the test 

case. 

Comment 6: The above automaton changes at the beginning of the test case (thus at t=0) to, 

for example, the “initialize light switch”. Simultaneously, the second, parallel automaton 

changes to the “light intensity constant dark” state. 

Comment 7: The “initialize light switch” state exits after 10 seconds have passed. The 

corresponding transition has the formal definition t >= 10 (cf. Figure 25). 

This way, it is very simple to show that the transition switches exactly at the time when t 

has become greater than or equal to 10. In this, the expression t describes the current 

time since the current state has been entered, i.e. it has a local context. 

Comment 8: The state “switch light switch to (on)” is depicted in dotted lines because it 

references another state (“initial light switch”) and performs the same operations as the 

original state.


Comment 9: The remaining process is now carried out similarly. The “switch light switch to” 

state exits again after 10 seconds. Since the transition switches into a final node, the test 

case ends at exactly this moment. 

Comment 10: The parallel, lower automaton switches at the time t=0 into the “light intensity 

constant dark” state. This state models how the ambient light intensity should behave for 

the test case. 

Informal process description of the test case in Figure 24: 

At the start of the test case, two parallel processes are activated: the process of the upper 

and that of the lower automaton. In both cases the process begins at the left-hand initial 

node. Since these nodes are not states and the test case may not ‘remain’ in such nodes, 

the test case switches directly via corresponding transitions into the “initialize light 

switch” and “light intensity constant dark” states. Since no action has been stored for the 

upper transition (cf. explanation 2 above), the states are “executed” immediately. Each 

state has a clearly defined behaviour: The “initialize light switch” state changes the 

channel “light_switch” to LIGHT_ON; the “light intensity constant dark” state sets the 

“light_intensity” channel to 0.0 at the earliest possible time. 

After this initialization the upper sub-automaton proceeds as follows: The “initialize light 

switch” state exits after exactly 10 seconds, if the transition “switch light condition: after 

10 s” (cf. explanation 7 above) becomes true. During the switching of the transitions, the 

actions within the transitions are executed. This transition does not define an action. Thus 

the test case now changes into the “switch light switch to on” state. This state does not 

change internal logic nor does it change channels because the “light_switch” is 

already set to LIGHT_ON. It thus does “nothing” and only waits for another 10 seconds to 

pass – up to the point when the last transition switches “end test after 10 s” and ends the 

test case by reaching a final node. 

The parallel lower automaton starts after the initialization in the “light intensity constant 

dark” state. As long as the automaton is in this state, the “light_intensity” is set to 

zero. The parameter t corresponds to the current time since the current state has been 

entered (also cf. comment 7 above). The transition leading from the “light intensity dark” 

state to the final node has no formal condition. Such a transition switches if the state 

ends from “within”. The fact of the matter, however, is that the ”light intensity constant 

dark” state does not end from within so that the transition never switches. This means: the 

automaton remains in the “light intensity constant dark” state forever. The lower 

automaton ends only if the upper automaton is terminated, since a parallel automaton in 

TPT is always terminated if (at least) one of its sub-automatons ends. 

Summary: 

For the above example the given explanations for the modelling language of TPT allow 

implementation of a test case in the TPT environment. The automaton mirrors the 

informal process description from Figure 23 in a formal manner 

2.2.5 The second test case: “lights on if light” 

The second test case “lights on if light” differs from the test case “lights on and dark” only 

in the sense of the light intensity of the surroundings. The test case is thus almost 

identical with the first test case from section 2.2.2 and therefore in the process modelling 

looks as follows:


Figure 25: Process modelling of the second test case 

The upper sub-automaton is unchanged. The only difference in the modelling of the two 

test cases consists in the definition of the lower sub-automaton: In the first test case 

(Figure 24) the “light intensity constant dark” state has been selected, while, by contrast, 

in the second test case (Figure 26) the “light intensity constant light” state is selected. 

If we compare these two test cases in a little more detail we will also find that the two test 

cases do not differ structurally. The distinction can solely be attributed to the definition of 

the behaviour – therefore the semantics– of the “light intensity constant” state. If we now 

assume that there was only one “light intensity” state for both test cases together (instead 

of “light intensity constant dark” for the first test case and “light intensity light” for the 

second), this state apparently has different semantics in the first and second test case, 

which results in different test cases. 

This is exactly what makes up the basic idea of TPT: The joint usage of a state makes it 

clear that the behaviour of the “light intensity” state has to be varied between the test 

cases, as defined by our descriptive or written form of the test cases. Here, it is important 

that this variation of the semantics does not take place locally for every individual test 

case, but is assigned globally to the state. This means: If there are test cases where the 

ambient light intensity varies between light and dark, the state does not have one but two 

(or even more) alternative semantics. Hence, for each specific test case one of these 

semantics must be selected at all points of the automaton at which the semantics are not 

clear (because there is more than one variant due to test-relevant aspects) in order to 

define clear, executable semantics of a test case. How this is done in detail will be 

explained further on. 

In other words, the following steps are required for the modelling of concrete test 

sequences: 

1. Modelling of the general process with the “standard semantics” of the states and 

transitions 

2. Modelling of the variants for the semantics of states and transitions at test-relevant 

places 

3. Generation of specific test cases by selecting and combining one variant each at all 

places of the automaton at which more than one variant exists 

We will now illustrate these considerations in an even more explicit manner with the help 

of the example of the “light intensity” state: In the above informal process description for


the first test case the behaviour of the light intensity has been defined as the equation 

light_intensity(t) = 0.0. This is the semantics for the “dark” case. The intensity 

does not vary. For the second test case a second “light” semantics must be modelled in 

the form of the light_intensity(t) = 100. In order to be able to distinguish these 

two variants of “light intensity” (i.e. without having to read the definition equations), they 

are given natural-language names: 

• “dark” light_intensity(t) = 0 

• “light” light_intensity(t) = 100 

If we look again at the depiction of the test sequences of the two test cases in Figure 24 

or Figure 26 we now see why the state “dark” or, respectively, “light” is depicted in red 

in the “light intensity” state: The state itself has the name “light intensity”. There are 

several alternative variants of the semantics for this state. Two of these variants are “light” 

and “dark”. 

Similarly we now find that in the upper sub-automaton of the example there are several 

elements for the alternative (marked red) definitions. In this case, there are two 

transitions; this means that alternative semantics are possible for transitions as well. States 

and transitions for which no red variants are depicted have only one semantic and thus 

require no addition to the explicitly selected variant. 

Besides variations in states there are also transition variations and so-called path 

variations. The tutorial part of this manual will not go into further detail concerning these 

types of variations, since the variation principle is similar to the state and transition 

variation. Still, these variation types are very important for the expression capability and 

the creative diversity of TPT test (cf. Figure 27). 

Figure 26: 

Definition of “TPT automaton” and “automaton instance” 

The unique feature of the automaton language of TPT is, as mentioned before, the fact 

that states, transitions and paths do not usually have any clear semantics but define 

several variants. Such an automaton is called TPT automaton. A TPT automaton itself, 

therefore, is not executable. Only the selection of the corresponding variants at all variant 

locations makes the semantics become clear and executable. Such a selection is called


automaton instance. Each automaton instance thus describes an explicit specification of 

the semantically ambivalent TPT automaton. 

2.3 Execution of the Modelled Test Cases 

In order to execute modelled test cases a few configurations need to be completed. 

2.3.1 Test set configuration 

Please select Execution -> Configure Test sets in order to compose tests for 

the execution. Click on the Add button for a new test set and select the test cases that 

shall be executed (cf. Figure 28). 

2.3.2 Execution Configuration 

Figure 27: Test set configuration 

Select Execution -> Execution configurations in order to select the tests that 

shall be executed in a certain environment. 

Click on the Add button to create a new execution configuration and select a test set 

specified before (cf. Section 2.3.1). Also a data dictionary where the test results shall be 

saved and a platform that has been configured before (cf. Section 2.2.1) need to be 

specified. May be only execution is selected but also assessments and report generation is 

possible (cf. Figure 29).


Figure 28: Execution configuration 

After clicking on the compile and run buttons the execution starts and the build window 

opens. The tests are executed and the TPT build window opens (cf. Figure 30). 

Figure 29: Build progress window


To explore the test results right-click on the scenario in the TPT build window and select 

view test results. Within the test data viewer the signals can be shown by dragging 

the signal into the graph (cf. Figure 31). 

2.4 Assessment 

Figure 30: View test results 

The assessment is programmed in Python under the assessment tab in the main window. 

A very simple example can be seen in Figure 32. The script in this example checks if the 

headlight is on during the state and when the default scenario is active. 

Figure 31: Simple assessment script


2.5 Report generation 

When the report check box is active in the execution configuration a test report will be 

generated from the test and assessment results. There is a default template for the report 

but it may be changed using the report template editor. The report can be opened after 

test execution and shows details of the test. It can be seen that the exported assessment 

results are explicitly shown in the report. Graphs in the report can be opened and zoomed 

in (cf. Figure 33). 

Figure 32: Test report example 

2.6 Executable platform lights control test 

A very basic but flexible platform configuration is the Executable platform configuration. 

This platform is any executable program that is specified. 

The executable must contain the loop of the TPT-VM and the system under test. An 

example, how this can be accomplished, can be found under “\examples\exeplatform”. 

Assume for example the lights control example is implemented as C-function that returns 

the “headlight” as function of “light_switch” and “light_intensity” 5 . In order 

to create a test-frame for this a test-frame needs to be programmed and compiled that 

calls periodically the API of the TPT virtual machine and the function 

5 unsigned char getHeadlightValue(unsigned char light_switch, double light_intensity);


“getHeadlightValue”. Templates and the API for this are available. Please refer to the 

lights control example C-code. 

Figure 33: Executable platform configuration

TPT Tutorial - PikeTec

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?