D.3.3 ALGORITHMS FOR INCREMENTAL ... - SecureChange

5.3 Evaluation with respect to the Success Criteria
In this section we evaluate the performance of the modeling and reasoning approach to
requirements evolution with respect to the success criteria identified in section 4.1.
SC1 : The phases of the change management process adopted by Air Navigation
Service Providers (ANSP) are identified. For each phase of the process the participants
and the artifacts used to support the phase are identified.
As attested by the graph illustrated in Figure 3, during the WS1 and WS2 workshops,
the requirement analysts gain a deep-understanding of the change management
process adopted by ANSPs. The process mainly consists of the following phases:
1. Brainstorming. This phase aims at exploring the different options for responding
to a proposed change and evaluating their feasibility. Air traffic controllers, technical,
and managers try to identify the new high level operational requirements and
the different possible alternative operational and technical procedures. The operational
requirements and procedures are represented as influence diagrams [13].
2. Risk Assessment. The objective of this phase is to identify the nature of the proposed
change. If there is no impact at operational level the change is minor and can
be directly implemented without performing any simulation or training for the air
traffic controllers. On the contrary, if simulation is needed, the change is a major
change.
3. Fast Time Simulation. The aim of this phase is to assess the impact of change on
the operational procedures by performing computer-based simulation.
4. Real Time Simulation. This phase evaluates the impact of the change with humans
in a real environment. Humans can propose minor changes to be implemented.
SC2 : The approach to model and reason on evolution can support and facilitate
the change management process adopted by Air Navigation Service Providers.
The participants P5 and P12 have pointed out that currently, in the whole ATM domain
an increasing interest is devoted to methodologies and processes supporting and
documenting the decision making activities within the SESAR Programme. They have
mentioned that main open issues are change management and the need of a formal
methodology to trace and assess the introduction of new operational concepts and their
impact on ATM Key Performance Areas. P5 and P12 have reported that now the different
phases of the change process adopted by Air Navigation Service Providers are supported
by influence diagrams that allow to trace strategic objectives to operational solution
and that allow to perform what if analysis to understand the impact of a proposed
change. P6 and P9 suggested that the Evolution Elicitation and Probability Estimation
might be useful during the brainstorming phase to identify the alternative operational
requirements associated with a proposed change. The Reasoning phase instead can be
used to support the decision makers in identifying the best solution at operational level
to be implemented.
The comments of P5 and P12, thus, indicate that the approach to model and reason
on evolution can support managers and controllers during each phase of the change
process.
SC3: The approach to model and reason on requirements evolution is sufficiently
expressive to capture the evolution in the ATM domain.
12