Testing Embedded Systems in the Automotive Industry with TTCN-3

Invest for the FutureThe contribution is partiallyfunded by the EFRE (Europeanfund for regional development)of the European Union.TTCN-3 User Conference 20093 – 5 June 2009 - ETSI, Sophia Antipolis, Francewww.temea.orgTesting Embedded Systems in theAutomotive Industry with TTCN-3Jens Grabowski (Universität Göttingen), Jürgen Großmann(Fraunhofer FOKUS), Friedrich-Wilhelm Schröer (Fraunhofer FIRST),Jacob Wieland (Testing Technologies), Diana Serbanescu(Technische Universität Berlin)grabowski@informatik.uni-goettingen.de,juergen.grossmann@fokus.fraunhofer.de, f.w.schroeer@first.fraunhofer.de,wieland@testingtech.com, diana.serbanescu@fokus.fraunhofer.de

Standardized Test Technology for the Automotive IndustryTEMEAWP1:AutomotiveRequirementswww.temea.orgWP5:Case StudyWP2:TTCN-3 embeddedConceptsWP4:Test Methodologyand QualityWP3:AUTOSARTest Framework

Motivationwww.temea.orgTesting software based embedded systems steadily increase incomplexity.In addition to that non-functional requirements, especially timerelated input-output behavior, have to be considered.Adequate and standardized test solutions are needed, which at leastfeature a minimum of flexibility, reusability and abstraction.3GOAL: Provide a standardized testing solution forstandardized development environments (e.g. AUTOSAR forAutomotive Solutions).GOAL: Tight Integration of real time testing concepts in anexisting test specification environment (i.e. The Test andTesting Control Notation)

Standardized Test Technology for the Automotive IndustryTTCN-3 embedded Taskswww.temea.org TTCN-3 embedded for real time systems TTCN-3 embedded for continuous behavior TTCN-3 embedded hybrid behavior Graphical presentation format for TTCN-3 embeddedPreparation for standardization4

Real Time Test System Requirementswww.temea.orgtest componenttiming checkvalue checktemporal controlstimulus : OUT_MSG_1[t max ]reaction : IN_MSG[t wait ]stimulus: OUT_MSG_2system under test (SUT)Standard: assessment of functional behavior (e.g. messagecontents).Additional: exact measurement, comparison and assessment ofmessage timing.Additional: temporal control of message dispatching.5

Real Time TTCN-3Simple Real Time Scenariowww.temea.orgtimer t1,t2;p_out.send(OUT_MSG_1);test componentsystem under testt1.start(t_max);stimulus - OUT_MSG_1(SUT)alt{time check[t MAX ][]p_in.receive(IN_MSG_1){setverdict(pass)};[]t1.timeout{setverdict(fail)}value check reaction - IN_MSG_1}time control [t WAIT ]t2.start(twait);t2.timeout;stimulus - OUT_MSG_2p_out.send(OUT_MSG_2);p_in.receive(IN_MSG_2);setverdict(pass); var reaction float - IN_MSG_2 r_time,s_time;p_out.send(OUT_MSG_1);s_time:=now;p_in.receive(IN_MSG_1)-> timestamp r_time;if(r_time>s_time+tmax) setverdict(fail);wait(r_time+twait);p_out.send(OUT_MSG_2);p_in.receive(IN_MSG_2);setverdict(pass);

Formalization of the Test Systemwww.temea.orgPa set of ports to communicate withthe System Under Test (SUT),a set Q of input queues to organize theorder of incoming messages,a set C of synchronized clocks tomeasure time and to simulate TTCN-3timers,a set M of messages,a set TP ⊆ TPdata ∪ TPtime of predicatesthat are used to characterize theproperties of incoming messages, anda set OP = {snap, check, enqueue,dequeue, first, encode, decode, match}of time-consuming operations that arenecessary to organize the handling ofmessages at ports.q 1..qnm 1..mnp 1..pn7

Example: Comparison of Message Timingin Standard TTCN-3www.temea.orgArrival of messages m o ,..,m n and thetimeout of timers t 0 ,..,t m are denotedby events e 0 ,..,e k+1 ,9timing is measured by comparisonof events, andonly events that occur in differentsnapshots are distinguishable.Duration between two consecutive snapshot denote the bestaccuracy of time measurement for standard TTCN-3. Theduration depends on:- the number of messages that arrive and the number of ports(queues) to check,- the duration of check, decode, match for individual messages wherethe duration of decode and match is directly dependent on thecontent and structure of the message under observation.

Worst Case Influence of Time ConsumingOperationswww.temea.orgProblematic Situations: message burst over one or multiple ports.Each alternative is defined by:Simple assumption: a new message has arrived at each port andnone of the messages match.10

Solutionwww.temea.org Seamless access to time Explicit measuring and access to the reception time ofmessages Utilities to handle comparison of time and temporalcontrol of statement execution11

Time: Concepts & Representationwww.temea.orgTime model based on positive real numbersActual time t = t( c0)can be directly obtained by the user (nowoperator).TTCN-3 Language Level:- now operator returnstime in seconds coded asa float value.- we allow arithmeticexpressions on timevalues- precision of timemeasurement can bespecified by means of theprecision annotationmodule{…var float myTimeVar;testcase myTc runs on myComp{…myTimeVar:=now+1.0;…}} with{precision:=0.001}

TTCN-3 embeddedMeasurement of Timewww.temea.org… to retrieve the enqueue time of a message,p.receive(t)-> timestamp myTime;// yields the reception time of a messageand time measurement at any place in the testvar float myTime:= now;// yields the actual time13

TTCN-3 embeddedVerification of Temporal Behaviourwww.temea.org Verification of enqueue time for incoming messages,procedure calls etc.p.receive(t)-> timestamp timevar {if (timvar>max){setverdict(fail)}else {setverdict(pass)}};14

TTCN-3 embeddedTemporal Controlwww.temea.org … at any place during test case execution,wait(timepoint); and similar for message timingwait(timepoint);p.send(t);15

TTCN-3 embeddedSettable Error Verdict Necessarywww.temea.org Double check the timing of test system behavior// test system to slowwait(timepoint);p.send(MSG_1);if(now >= timepoint + tol) setverdict(error);// SUT to slowwait(timepoint);p.send(MSG_1);if(now >= timepoint + tol) setverdict(fail);// SUT or test system to slowwait(timepoint);p.send(MSG_1);if(now >= timepoint + tol) setverdict(inconclusive);16

Use Case: Test of an IndicatorTesting Temporal Propertieswww.temea.org Maximum activation time 60 ms, phase length 600 ms Synchronization between signals: distance < 5 ms17

Use Case: Test of an IndicatorTesting Activation of Indicatorwww.temea.orgtestcase tc1( ) runs on IndicatorTestComponent{var float l_actv, r_actv, f_actv;const float TMAX = 0.06;activate(tc_timout);leverIn.send (LEFT);l_actv:= now;interleave{[ ] FrontOut.receive(ON) -> timestamp f_actv;[ ] RearOut.receive(ON) -> timestamp r_actv;}if ((f_actv-l_actv > TMAX)or (f_actv-r_actv > TMAX)){setverdict(fail)}setverdict(pass);}18

Use Case: Test of an IndicatorTesting Signal Synchronizationwww.temea.orgtestcase tc2( ) runs on IndicatorTestComponent{var float r_actv, f_actv;const float TMAX = 0.005;activate(tc_timout);leverIn.send (LEFT);interleave{[ ] FrontOut.receive(ON) -> timestamp f_actv;[ ] RearOut.receive(ON) -> timestamp r_actv;}if (abs(r_actv-l_actv) > TMAX){setverdict(fail)}setverdict(pass);}19

Summary and Outlookwww.temea.orgRT concepts are tightly integrated with TTCN-3 and- provide means for an exact measurement, comparison and verificationof the timing of incoming messages, and- enables the detection of timing problems during test execution andmessage dispatchingImplementation of ConceptsIntegration with high level modeling techniques (i.e. declarativeapproaches to specify timing constraints).Definition of coding and design guidelines to support the RTcapabilitiesof the newly introduced TTCN-3 concepts.20

Contact and Infowww.temea.orgwww.temea.orgContactJürgen GroßmannFraunhofer FOKUSKaiserin-Augusta-Allee 31,10589 BerlinTel: +49-30-3463-7390juergen.grossmann@fokus.fraunhofer.de