ETTC'2003 - SEE

eference books, but it is as fundamental for test data or for any test. Henceforth it is
necessary to document all test conditions, as well as all decisions that led to freeze this or that
freedom degree within the scenario or the environment.
This ultimate remark is important, as the VV&A and VV&C processes are very often
described are costly, or as a necessary evil that generates initial overhead costs which lead to
cost-efficiency only through subsequent reuse. However the same argument should be used
stricto sensu for tests and test data, which is rarely underlined, all the more since in the past
decades with the high budgets it was possible to conduct almost any desired test: we have
today lots of test data that are totally useless as they cannot be reused in a coherent manner.
As a first idea of the induced costs, we refer to the report “prefeasibility study on
simulation based design and virtual prototyping”, published by the NATO Industrial Advisory
Group in September 2000, reference NIAG-D(2000)9 AC/141(NG-6)D/25 : the VV&A cost
is estimated as 15% of the global acquisition cost, and between 2 and 6% for a reused
simulation.
Back to verification and validation, it is difficult to evaluate a priori the necessary
validation level, even if it is possible to define the necessary effort to reach a given credibility
level. This comes from the fact that validation is performed in relation with an intended use of
the product (data, model or simulation), and this cannot be made explicit from the beginning
of the life cycle. Such a difficulty should not be seen as a tremendous obstacle, but as the
necessity to conduct VV&A and VV&C within an iterative an incremental approach,
throughout the whole system lifecycle.
Technologies : solutions, constraints and necessary effort
Virtual proving grounds, with a tight coupling between simulations and test data
within virtual and digital synthetic environments, need to follow a few constraints in order to
be viable. There is a need for interoperability between all simulations, even between systems
which rely heavily on software, like C4ISR systems. This will be always more frequent with
the development of systems-of-systems and network-centric warfare or cooperative
engagement issues.
It is not reasonable to develop every time specific interfaces, as a trivial cardinality
analysis shows: their number is one order of magnitude larger that the number of incriminated
systems, and an iterative evolution of the system implies thus an evolution of all interfaces,
i.e. the order of magnitude of all considered connected systems.
The (only) cost-efficient way is to adopt reference architectures at the various levels,
what is usually known as meta-models from which the conceptual models of the systems can
be easily established. To follow these reference architectures is indeed a constraint, but much
smaller than the savings it generates, since the evolution of a system needs a priori only the
modification of the interface with the reference model.
Such reference architectures exist: HLA for simulations, others are under development
for C4ISR systems, or are foreseen for robotic systems (cf. JAUGS, “Joint Architecture for
Unmanned Ground Systems”, promoted by the US Army)…
In a similar fashion, at data level, exchange standards are essential. As there does not
exist a priori a unique format (SEDRIS should be a major format in the coming years), there
is a need for meta-data standards. This is under study, for instance by the OMG, the “ object
management group”.
A quick review of the main efforts that should be investigated in order to optimize the
collaboration between tests and simulations, one finds: