Accident risk assessment for advanced ATM - ATM Seminar

Accident risk assessment for advanced ATMH.A.P. Blom, G.J. Bakker, P.J.G. Blanker, J. Daams,M.H.C. Everdij and M.B. KlompstraNational Aerospace Laboratory NLRPO Box 90502, 1006 BM AmsterdamE-mail: blom@nlr.nlAbstractBy now, safety is recognised as a key quality on whichto select/design advanced ATM concepts, even when capacityand efficiency are the drivers of the development. Thesafety target is often described as ‘equal or better’ in comparisonwith existing practice, allowing a large freedom inhow safety is expressed, let alone measured. In effect, newCNS/ATM concept developments are typically accomplishedwithout the use of feedback from appropriate safety assessments.ATM concept design teams (e.g. of Free Flight, or4D-ATM) try to realise capacity-efficiency enhancements byexploiting new technology, changing human controller rolesand introducing new procedures, while relying on the establishedsafety-related indicators in ATM such as conflict ratesand types, workload of human operators and failure ratesand effects of technical systems.ATM, however, is the result of complex interactions betweenmultiple human operators, procedures and technicalsystems, all highly distributed. This yields that providingsafety is more than making sure that each of the ATM elementsfunctions properly safe; it is the complex interactionbetween them that determines safety. The assessment of isolatedindicators falls short in covering the complex interactionsbetween procedures, human operators and technicalsystems in safety-critical non-nominal situations. In orderto improve this situation, this paper outlines a novel probabilisticrisk assessment methodology which has specificallybeen developed for application to ATM. In addition, thispaper presents risk assessment results which have been obtainedwith this approach for two en-route streams of RNP1equipped traffic flying in opposite direction within two conventionalATM concepts and two airborne separation assurancebased concepts. These results illustrate that our newmethodology supports safety-based ATM design.I. INTRODUCTIONATM is the result of complex interactions between humanoperators, procedures and technical systems (hardwareand software), all highly distributed. Providing safetyis more than making sure that each of these elements functionsproperly safe. The complex interactions between thevarious elements of ATM significantly determine safety.Therefore it is imperative to understand the safety impactof these interactions, particularly in relation with non-nominalsituations. Traditional ATM design approaches tendfirst to design advanced ATM that provides sufficient capacity,and next to extend the design with safety features.The advantage of this approach is that ATM developmentscan be organised around the clusters of individual elements,i.e. the communication cluster, the navigation cluster, thesurveillance cluster, the automation tools cluster, the HMIs,the advanced procedures, etc. The key problem is thatsafety effects stay unclear. A far more effective approach isto try to design an ATM system that is inherently safe at thecapacity level required. From this perspective, safety assessmentshould be one of the primary filters in ATM conceptdevelopment. An early filtering of ATM design conceptson safety grounds can potentially avoid that a costlydevelopment programme turns out ineffective, or that aneven more costly implementation programme fails. Althoughunderstanding this idea is principally not very difficult,it can only be brought into practice when an ATMsafety assessment approach is available that provides appropriatefeedback to the ATM designers already at an earlystage of the concept development (figure 1).ATMdesignSafety / CapacityAssessmentFig. 1. Safety feedback based ATM design

This feedback should not only provide information onwhether the design is safe enough, it should also identifythe safety-capacity bottle-necks. By now, consensus isbuilding that appropriate ATM safety modelling approachesare needed to understand the mechanisms behind designingadvanced ATM. It is also recognised that, oncesuch an ATM safety modelling approach is available, asafety feedback based design approach of future ATM willbecome feasible (Haraldsdottir et al., 1997; Odoni et al.,1997; EVAS, 1998).Safety is a general notion, which is typically studiedfrom one of three different perspectives: Safety perception (e.g. by pilot, controller, passenger,human society, etc.). An ATM design that is perceivedas being unsafe will not easily be accepted by the humansinvolved. Fact is that a positive perception aboutthe safety of an ATM design is an implementationcriticalrequirement. By its very nature, however, safetyperception is a subjective notion, and therefore insufficientto really approve safety-critical changes inATM. Dependability of a technical system (e.g. of a computerprogram, an aircraft navigation system, a satellitebased communication system, etc.). Dependabilitymetrics are definitively objective. They are widelystudied in literature (e.g. Randell, 1995; DAAS, 1995).However, they have been developed to cover technicalsystems only (e.g. SAE, 1994, 1995; EATCHIP,1996), and not the human operators and procedures ofATM (Klompstra and Everdij, 1997).Accident risk (e.g. for 1st, 2nd and 3rd parties in airtransport) metrics definitively are objective and arecommonly in use for other human controlled safetycriticaloperations such as chemical and nuclear industries(Royal Society, 1983). Two well known ICAOadopted accident risk metrics are for collision of anaircraft with another aircraft during en-route phase,or with fixed obstacles during landing. A recent reviewof various accident risk metric possibilities in airtransport is given in (Moek et al., 1997).In view of the ATM safety assessment needs, the accidentrisk perspective has the best joint characteristics: 1) Itimplies the use of objective risk metrics, 2) It has provenits usability to human controlled safety-critical operations,and 3) It is supported by ICAO. As such, in this paper ATMsafety will be considered from an accident risk perspective,with emphasis on risk of collision between two aircraft.For air traffic the fatal accident risks should be of theorder of 10 7 -10 10 per aircraft flight hour. To developsome feeling of the difficulty to assess such rare events,it is quite helpful to understand why the well known fasttime simulators like NASPAC, RAMS or TAAM fall shortfor that purpose. One major shortcoming of these tools isthat they are not really capable of modelling the aviationsafety-critical combinations of non-nominal events, theyoften do not even model the single non-nominal events.Another major shortcoming is that an accident rate of, say,10 9 per aircraft flight hour can not in a practically reasonableway be reached through a straightforward simulation,since this would require a simulation of 10 10 aircraft flighthours. This problem is well illustrated by the ATM safetyiceberg (figure 2). To assess a catastrophic accident rate,one really needs to decompose the risk assessment probleminto an effective hierarchy of simpler conditional assessmentproblems, where simplicity means an appropriatecombination of scope (e.g. volume of airspace) and depth(i.e. level of model detail) at each conditional assessmentlevel. Indeed, tools like TAAM apply to assessments thataddress a broad scope in combination with a low level ofnon-nominal detail.Assessment approachAccident Risk ModellingDependabilitymodellingFast-timesimulationReal-timesimulationFig. 2. ATM safety icebergEventsCatastrophic accidents (≈10 -9 /fl.hr.)Technical failures (≈10 -4 /fl.hr.)ATCo actions (≈10 /fl.hr.)Pilot actions (≈100 /fl.hr.)In general, the accident risk assessment problem hasbeen widely studied for other safety-critical operations,such as the nuclear and chemical industries, and for theseapplications, numerous techniques and tools have been developed.In order to take maximal advantage of this existingbody of knowledge, we made a thorough study of theapplicability of these techniques to accident risk assessmentin air traffic (Everdij et al., 1996a). A large varietyof techniques has been identified, varying from qualitativehazard identification methods such as Preliminary HazardAnalysis (PHA), Common Cause Analysis (CCA) andFailure Mode and Effect Analysis (FMEA), through staticassessment techniques such as Fault Tree Analysis (FTA)and Event Tree Analysis (ETA), to dynamic assessmenttechniques such as Petri net and Markov chain modelling,

A 10/T localA 0AircraftmissionA 9A 14CPA 3Pilot NotFlyingA 1FlightplanaircraftA 2Pilots SkillA 4Pilot FlyingA 6DisplayaircraftA 15A 16A11FPCM other FPCM ownSTCD&RA 9 A 9A17A 12FlightplanADS-Bdata otheraircraftA 9A 1*withoutATCocurrentATC+ Initial Free Flight equipped- TLS (Eurocontrol)xTargetFreeFlightequippedA 10/T localA 0AircraftmissionA 9A 14CPA 3Pilot NotFlyingA 1FlightplanaircraftA 2Pilots SkillA 4Pilot FlyingA 6DisplayaircraftA 15A 16A11FPCM other FPCM ownSTCD&RA 9 A 9A17A 12FlightplanADS-Bdata otheraircraftA 9A 1dynamic event trees, etc. (Aldemir et al., 1994). Eachof these techniques has advantages and disadvantages, butthese appear to be minor in comparison to what is requiredfor modelling ATM related risk. The key finding is thatthe established techniques fail to support a systematic approachtowards modelling stochastic dynamical behaviourover time for complex interactions of highly distributedATM (see figure 3).Potential fatalitiesThousandsHundredsTensLocalisedinteractionsDistributedinteractionsHighlydistributedinteractionsFig. 3. Potential fatalities and distribution level of ATM andother safety critical activities.The established techniques would therefore force one toadopt a rather heuristic type of argumentation in trying tocapture the complex interactions inherent to ATM.The basic ATM safety assessment needs have alreadybeen identified in (Blom, 1992b). This finding motivatedthe development of an adequate safety assessment approachwithin a project named TOPAZ (Traffic Organization andPerturbation AnalyZer). The scientific basis for this wasthe idea to explore a stochastic analysis framework (Blom,1990) which supports stochastic models where both discreteand continuous variables evolve over continuous time,possibly affected by probabilistic disturbances, and theknowledge that this framework would be sufficiently generalto properly model and evaluate ATM safety problems.In the mean time, from parallel conducted studies onadvanced ATM it became crystal clear that without an appropriateaccident risk model it would be difficult to evermanage a cost-effective design of advanced ATM. In thesestudies three complementary perspectives have been considered:1) the selection of route structures perspective(Blom and Bakker, 1993), 2) a stochastic dynamical gameperspective (Blom et al., 1994) and 3) an ATM overall validationperspective (Blom et al., 1995).The accident risk assessment results obtained throughstochastic analysis studies have initially been exploited foran RLD/LVB project towards the assessment of accidentrisk for staggered landings on converging runways (Bakkeret al., 1995; Everdij et el., 1996c). All this contributed tothe development of both the TOPAZ assessment methodology,and a growing suite of TOPAZ tools. In this paper,emphasis is on the former, for the reason that an effectiveusage of the suite of tools requires firm background in thenovel methodology.Recently, by a joint effort of Eurocontrol and FAA, incollaboration with some key developers of aviation riskassessment tools, an overview has been produced that outlinesthe relevant approaches currently in development and/or in use for the safe separation assessment of advancedprocedures in air traffic (Cohen et al., 1998). In additionto TOPAZ, four other collision risk directed approaches,ABRM, ASAT, ICAO’s Collision Risk Model (CRM) andRASRAM (Sheperd et al., 1997), have been identified andreviewed; TOPAZ appeared to be most advanced in goingbeyond established approaches.This paper is organised as follows. Section II gives anoverview of the methodology. Next, section III outlines theprinciples of the underlying stochastic dynamical framework.Section IV presents for several RNP1 example scenariosthe results of TOPAZ based risk assessments. SectionV gives concluding remarks on the methodology. Thepaper ends with references and acronyms.II. THE TOPAZ METHODOLOGYThe TOPAZ methodology has been developed to providedesigners of advanced ATM with safety feedback followingon a (re)design cycle. An illustrative overview ofhow such safety feedback is obtained during a TOPAZ assessmentcycle is given in figure 4.R isk (in ac cid en ts/flig h th o ur)10 -510 -610 -710 -810 -910 -1010 -11* Aircrew overload* Crew hindrance* Pilot reaction too late* Pilot ignores conflict* Crew disagreement* Reduced visibility* ...Risk vs. route spacing-11010 -2- TLS (Eurocontrol)* without ATC10 -3o current ATC+ initial Free Flightx extended Free Flight10 -4ta rget0 5 10 15 20 25 30Spacing S (in km)1THij ij[ 0 ] ∑∑∑∫ ϕ, ( tκ T0tH*lijκ ) dt P{ κt*lκ }R = = ⋅ =2ij≠ilFig. 4. TOPAZ assessment cycle

actions are the product of human internal states, strategiesand the environment. By now, it is a widely accepted belief(Amalberti and Wioland, 1997; Hollnagel, 1993; Bainbridge,1993) that for the modelling of the human the establishedHuman Reliability Analysis (HRA) techniquesfall short for complex situations, and that one should ratheraim for contextual performance models that are based ongenerally-applicable human cognition and responsibilityprinciples. It should also be noticed that the in HRA widelyused skill-, rule- and knowledge-based errors (Reason,1990) essentially fall short to pay proper respect to, forexample, situations where the operator chooses to let aneven more urgent problem receive attention when the subjectivelyavailable time is short or when high workloadcauses one to make quick decisions, without bothering excessivelyabout the quality of those decisions. It should benoticed that these effects are inextricably bound up withhuman flexibility and the ability of humans to deal withunforeseen situations. When assessing ATM safety, it isnecessary to take these aspects of human performance intoaccount.The main benefits expected from contextual models isthat they provide better feedback to designers and that theyremove the need to use overly conservative individual submodelsfor relevant operator actions that may blur understandingof how safety is achieved in ATM. In order todevelop appropriate models for this, mathematicians andpsychologists are jointly developing high-level models ofcognitive human performance, through a sequence of studies(e.g., Biemans and Daams, 1997; Daams and Nijhuis,1998). At this moment this collaboration has led to a novelcontextual human task-network model, which is formulatedin terms of a DCPN, and which effectively combinesthe cognitive modes of Hollnagel (1993) with the MultipleResources Theory of Wickens (1992), the classicalslips/lapses model (Reason, 1990) and the human capabilityto recover from errors (Amalberti and Wioland, 1997).In addition, we have developed a model for the evolutionof situational awareness errors. Compared with those consideredin a recent study by (Hart et al., 1997), our approachshows to be an innovative one.c. Perform stochastic analysisAlthough it definitively is possible to realise a straightforwardMonte Carlo simulation of the SDE model, it willbe clear from the earlier discussion that this will not bereally effective for the assessment of catastrophic risks inaviation. In order to develop an effective approach to thenumerical evaluation of an SDE model, the SDE shouldbe analysed first by mathematicians with the appropriatebackground in the theory of stochastic analysis. At thismoment this is done on a case by case basis. For eachcase the aim is to analyse the SDE model such that itsnumerical evaluation can be done by decomposition intoa logical sequence of fast-time simulations, Monte Carlosimulations and/or analytical evaluations. The aim alwaysis to first decompose the risk assessment problem into severalconditional assessment problems for which appropriateassessment techniques are available or feasible. Themain principle we are using for identifying an appropriatedecomposition is the following: under quite general conditions,the solution of an SDE is a strong Markov process.This means that the Markov property also holds true forstopping times (sometimes called Markov times). Thesestopping times serve as the mathematical powertool to decomposethe risk assessment for an SDE model. So far thisapproach appeared to work satisfactorily for all situationsevaluated.d. Execute the various assessment activitiesTypically, the resulting sequence of conditional assessmentsreadsasfollows:1. Run a conventional fast time simulation (e.g. withTAAM) to identify traffic densities and encounter typefrequencies.2. Input these traffic densities and encounter type frequenciesto a safety-directed human simulator to identifyappropriate pilot and/or controller characteristics.3. Input these conditional human characteristics to aMonte Carlo simulation that identifies and statisticallyanalyses critical conditional events, such as incidents.4. Input these critical conditional event characteristicsto a Monte Carlo simulation that identifies potentialaccident characteristics.5. Input these potential accident characteristics to a conditionalcollision risk analyser.6. Transform all results from the preceding conditionalassessments into appropriate safety metrics.7. Identify the safety-separation and/or safety-modellingbottlenecks, of the specifically modelled ATM concept/scenario.For each of these activities, except 1., dedicated computertools have been and are being further developed withinthe TOPAZ project. The splitting of activities 3, 4 and 5,from each other usually appears to be the most challengingone, for the very reason that often there are many dependenciesbetween various elements of a hazardous air trafficsituation. In order to handle this in a valid way, we makeuse of a mathematical framework, the basis of which isexplained in section III.

e. Validation of the risk assessment exerciseA crucial issue concerns the validation that a risk assessmentexercise is performed to an acceptable degree,without the need to first employ very expensive large scalereal time simulations of new concepts. Due to our underlyingstochastic analysis framework, such a validation canbe done through executing the following activities: Judge the level of conservatism of the assumptionsadopted for the development of the DCPN instantiationfor the situation considered. This should be donethrough active involvement of operational and designexperts. Verify the correctness of the instantiated DCPN versusthe results of the qualitative assessment and theassumptions adopted. This should be done by stochasticanalysis TOPAZ experts, with at least one who hasnot been involved with the DCPN instantiation. Verify the correctness of the mathematical transformationsapplied to the instantiated stochastic dynamicalmodel. This should be done by applying mathematicaltools from stochastic analysis theory. Verify that the various assessment activities have beenexecuted according to the unambiguous mathematicalmodel developed, including the decomposition. Thisshould be done by stochastic analysis experts.III. THE MATHEMATICAL FRAMEWORKEach DCPN instantiation can be represented by an SDEon a hybrid state space (Everdij and Blom, 1998), whichhas a strong Markov process f t g on a hybrid state spaceas its unique solution. The hybrid state process f t g hastwo components, i.e. t = (x t ; t ); with x t the componentassuming values in a Euclidean space and with t the componentassuming values in a discrete space. From the theoryof Markov processes it then follows that it is possibleto characterise the evolution of the density-distributionp t () of the joint process through a well-defined differentialequation in function space:ddt p t() =Lp t ()with L an operator defined by the Markov process f t g.Due to the strong Markov property, this differential equationalso applies under the condition of an f t g-adaptedstopping time (also referred to as Markov time):dp dt () =Lp tj tj (); for t>:It is particularly relevant to notice that the above equationsare well known for Markov chains, i.e. Markov processeswith discrete state space, which processes have shown tobe very useful in the development of advanced dependabilityand performability assessment methodology (e.g.Pattipati et al., 1993; Fota et al., 1997). For hybrid stateMarkov processes, this equation is well known in Bayesianestimation theory (e.g. Blom, 1990) and this has a.o. ledto advanced multi target multi sensor tracking applications(e.g. Blom et al., 1992a).The above equations imply that once the scenario to beassessed on collision risk has been represented through aDCPN instantiation, all probabilistic properties are welldefined,including the collision risk. Let yt i and vt i be thecomponents of x t that represent the 3D location and 3Dvelocity of aircraft i, i 2 f1;::: ;ng.Let y ijtlet v ijt= y i ty j t ,= v i tv j and let t Dij be the area such that y ijt2 D ijmeans that at moment t the physical volumes of aircraft iand j are not separated anymore (i.e. they have collided).Each time the process fy ijt g enters the area Dij , we say anincrossing occurs, and each time the process fy ijt g leavesthe area D ij , we say an outcrossing occurs. The first incrossingfor the pair (i; j) is a collision for that pair. If weassume that the relative speed v ijtis very rapidly going tozero as long as y ijtresides in D ij , the chances are zero thatthere is more than one incrossing per aircraft pair, and thusthe expected number of incrossings equals the expectednumber of collisions. Following (Bakker and Blom, 1993)the expected number R [0;T ] of incrossings, or collisions,between aircraft pairs in the time-interval [0;T]satisfies:R [0;T ] =nXnXi=1 j>iZ T0' ij (t) dtwith ' ij (t) the incrossing rate, which is defined by:' ij (t) = lim P fy ijt#0=2 D ij ;y ijt+ 2 Dij g=In (Bakker and Blom, 1993) it is also shown that ' ij (t)is well-defined, and can be evaluated under non-restrictiveassumptions as a function of the probability density of thejoint relative state (y ij, t vij t). In general, a characterisationof this probability density is complex, especially sincethere are combinatorially many types of non-nominal e-vents. A plausible way out of this is by conditioning onclasses of non-nominal events, where those non-nominalevents are placed in the same class if they have a similarimpact on the subsequent evolution of the relative stateprocess fy ij , t vij tg. This is done through 1) defining an appropriateevent sequence classification process f t g,suchthat the joint process f t ; t g is a strong Markov processas well, and 2) subsequently identifying an appropriatef t ; t g-adapted stopping time ij such that there is a zero

probability that the pair (i; j) collides before ij . Withthis, the above equations can be transformed into:R [0;T ] =P f ij ijnXi=1 j>inX X= gZ T' ij (t j ij= ) dt ij ijwith ' ij (t j ij= ) the conditional incrossing rate, ijbeing defined for t ij by:' ij (t j ij ij = ) =lim P fy ijt t =2 Dij ;y ijt+ 2 Dij j ij= g=#0 ijIn figure 5, the equation for R [0;T ] is presented R in the formof a tree, in which f ij T() is short for ' ij (t ij j ij= ij) dt P f ij= g. This tree has some resemblance with ijthe well known fault tree. However, due to the underlyingstochastic and physical relations, our new tree differssignificantly and is named Collision Risk Tree.R [0;T ]j+f ij ()j j jR 'ij(tj) dt PfgFig. 5. Collision Risk TreeFor the quantification of the boxes in the collision risktree, use is made of three types of evaluations: Monte Carlo simulations of the DCPN to quantifyP f ij= g and the statistical properties of the ijrelevant DCPN components at the stopping time ij . Evaluations of the evolution of the relative aircraftstates from stopping time ij on, and for each ij= ij. When complexity requires, this process can evenbe done for a sequence of R increasing stopping times.T Numerical evaluation of ' ij (t ij j ij = ) dt, ijusing the Generalized Reich equation of (Bakker andBlom, 1993), see also (Kremer et al., 1998).IV. RNP1 IN CONVENTIONAL AND AIRBORNESEPARATION ASSURANCE SCENARIO EXAMPLESIn this section, the TOPAZ approach is used to evaluatea simple scenario of two en-route traffic streams ofRNP1 equipped traffic, flying in opposite direction, all atone single flight level. This rather hypothetical scenariohas been developed by Eurocontrol with the aim to learnunderstanding how ATC influences accident risk, and howfar the nominal separation S between opposite RNP1 trafficstreams can safely be reduced. The specific details ofthis scenario are (Everdij et al., 1997a): Straight route, with two traffic lanes (figure 6), Flight plans contain no lane changes Parameter S denotes distance between the two lanes, Opposite traffic flows along each lane, Aircraft fly at one flight level only Traffic flow per lane is 3.6 aircraft/hour, All aircraft nominally perform RNP1, None of the aircraft are TCAS equipped, Target level of safety is 5 : 10 9 accidents/flight hour.This simple scenario is considered for the following fourATM concepts:A) Procedural separation only. In this case there is noATC surveillance system. This is the type of situationencountered with traffic over the North Atlantic.B) STCA-only based ATC. In this case there is radarbased surveillance and R/T communication, but it isassumed that ATC is doing nothing unless its STCAsystem issues an alert; thus assuming no monitoringby the ATCo. It should be noticed that this differs significantlyfrom conventional ATC, where an executivecontroller autonomously monitors and issues correctiveactions, while STCA is a safety net only.C) Basic airborne separation assurance. In this casethere is ADS-B surveillance and R/T between aircraft,but there is no ATC. For this concept it is assumed thataircraft behave co-operatively, in the sense that whenan aircraft’s CDR (Conflict Detection and Resolution)system detects a conflict with another aircraft, then itspilot will try to make an avoidance manoeuver. Thus,in most cases both pilots will try to make an avoidanceFig. 6. Opposite direction traffic in a dual lane routeS

[16] J. Daams, G.J. Bakker and H.A.P. Blom, Safety evaluation ofencounters between free-flight equipped aircraft in a dual routestructure, NLR report, forthcoming, 1998b.[17] J. Daams and H.B. Nijhuis, Human Operators Controllability ofATM safety, ARIBA, NLR final report, forthcoming, 1998.[18] DAAS (Dependability Approach to ATM Systems), Work packagereports for the European Commission DG XIII, 1995.[19] M.H.A. Davis, Piecewise Deterministic Markov Processes: a generalclass of non-diffusion stochastic models, J. Royal Statist. Soc.(B), Vol 46, pp. 353-388, 1984.[20] EATCHIP, Air Navigation System Safety Methodology, Eurocontrol,Edition 0.4, Working Draft, 1996.[21] R.J. Elliott, Stochastic calculus and applications, New York,Springer, 1982.[22] EVAS, EATMS Validation Strategy Document, Edition 1.1, Eurocontrol,June 1998.[23] M.H.C. Everdij, M.B. Klompstra, H.A.P. Blom and O.N. Fota,Evaluation of hazard analysis techniques for application to enrouteATM, MUFTIS Final Report on Safety Model, Part I, NLRreport TR 96196 L, 1996a.[24] M.H.C. Everdij, M.B. Klompstra and H.A.P. Blom, Developmentof mathematical techniques for ATM safety analysis, MUFTISFinal report on Safety model, Part II, NLR report TR 96197 L,1996b.[25] M.H.C. Everdij, G.J. Bakker and H.A.P. Blom, Application ofCollision Risk Tree Analysis to DCIA/CRDA through support ofTOPAZ, NLR report CR 96784 L, 1996c.[26] M.H.C. Everdij, G.J. Bakker, H.A.P. Blom and P.J.G. Blanker,Demonstration report in preparation to Designing EATMS inherentlysafe, TOSCA II WP4 phase I report, NLR, 1997a.[27] M.H.C. Everdij, H.A.P. Blom and M.B. Klompstra, DynamicallyColoured Petri Nets for Air Traffic Management Safety purposes,Proc. 8th IFAC Symposium on Transportation Systems, pp. 184-189, 1997b.[28] M.H.C. Everdij and H.A.P. Blom, Piecewise Deterministic MarkovProcesses represented by Dynamically Coloured Petri Nets,Submitted, 1998.[29] N. Fota, M. Kaaniche and K. Kanoun, A modular and incrementalapproach for building complex stochastic Petri net models. Proc.First Int. Conf. on Mathematical Methods in Reliability, 1997.[30] A. Haraldsdottir et al., Air Traffic Management Concept BaselineDefinition, NEXTOR Report RR-97-3, Boeing, 1997.[31] S. Hart et al. A designers guide to human performance modelling,AGARD AMP Working Group 22 draft report, 1997.[32] J.M. Hoekstra, R.C.J. Ruigrok and R.N.H.W. van Gent, Conceptualdesign of Free Flight Cruise with Airborne Separation Assurance,NLR report TP 98252, 1997.[33] E. Hollnagel, Human Reliability analysis, context and control.Academic Press, London, 1993.[34] M.B. Klompstra and M.H.C. Everdij, Evaluation of JAR andEATCHIP safety assessment methodologies, NLR report CR97678 L, 1997.[35] H.J. Kremer, G.J. Bakker and H.A.P. Blom, Geometric and probabilisticapproach towards conflict prediction in free flight, forthcoming,1998.[36] G. Moek, M.B. Klompstra, H.A.P. Blom et al., Methods and Techniques,GENOVA Final Report, NLR, 1997.[37] A.R. Odoni et al., Existing and required modeling capabilities forevaluating ATM systems and concepts, Final report, MIT, March1997.[38] K.R. Pattipati, Y. Li, and H.A.P. Blom, A unified framework forthe performability evaluation of fault-tolerant computer systems,IEEE Transactions on Computers, Vol. 42 (1993), pp. 312-326.[39] B. Randell (Ed.), Predictably dependable computing systems,Springer, 1995.[40] J. Reason, Human error, Cambridge Univ. Press, 1990.[41] Royal Society, Risk assessment, report of a Royal Society StudyGroup, 1983[42] SAE, ARP 4761, Guidelines and methods for conducting thesafety assessment process on civil airborne systems and equipment,S-18 Committee, Society of Automotive Engineers, Inc.,March 1994.[43] SAE, ARP 4754, Certification considerations for highlyintegratedor complex aircraft systems, Systems Integration RequirementsTask Group AS-1C, Avionics Systems Division, Societyof Automotive Engineers, Inc., Sept. 1995.[44] R. Sheperd, R. Cassell, R. Thava and D. Lee, A reduced aircraftseparation risk assessment model, Proc. AIAA Guidance, Navigationand Control Conf., New Orleans, August 1997.[45] R.N.H.W. Van Gent, J.M. Hoekstra and R.C.J. Ruigrok, FreeFlight with Airborne Separation Assurance, Proc. CEAS symposium,October 1997, Amsterdam.[46] C.R. Wickens, Engineering, psychology and human performance,Merrill, 19924DABRMADS-BASATATCATCoATMCCACDRCNSCRMDAASDCPNEATCHIPETAFMEAFTAHMIHRAICAONASPACNLRNMPHARAMSRASRAMRNP1R/TSDESTCATAAMTCASTOPAZACRONYMS4-DimensionalAnalytic Blunder Risk ModelAutomatic Dependent Surveillance-BroadcastAirspace Simulation and Analysis for Terminalinstrument proceduresAir Traffic ControlAir Traffic ControllerAir Traffic ManagementCommon Cause AnalysisConflict Detection and ResolutionCommunication, Navigation and SurveillanceCollision Risk ModelDependability Approach to ATM SystemDynamically Coloured Petri NetEuropean Air Traffic Control Harmonisation andIntegration ProgrammeEvent Tree AnalysisFailure Mode and Effect AnalysisFault Tree AnalysisHuman Machine InterfaceHuman Reliability AnalysisInternational Civil Aviation OrganisationNational Airspace Systems Performance AnalysisCapabilityNationaal Lucht- en RuimtevaartlaboratoriumNautical MilePreliminary Hazard AnalysisReorganized ATC Mathematical SimulatorReduced Aircraft Separation Risk Assessment ModelRequired Navigational Performance (95% of timewithin 1 NM)Radio TelephonyStochastic Differential EquationShort Term Conflict AlertTotal Airspace and Airport ModellerTraffic alert and Collision Avoidance SystemTraffic Organization and Perturbation AnalyZer