Brief on Missile Defense Agency Modeling and Simulation - National ...

Missile Defense Agency
Modeling and Simulation
25 Sept 2012
Presented to:
NDIA (ICOTE) Industrial Committee on Test & Evaluation
Keith Englander
Director for Engineering
Approved for Public Release
12- MDA-7017 (24 September 12)
Missile Defense Agency

XX/XX/07 2
BMDS Evaluation Strategy
“The BMDS performance evaluation strategy is to
develop models and simulations of the BMDS and
compare their predictions to empirical data collected
through comprehensive flight and ground testing to
validate their accuracy, rather than physically testing all
combinations of BMDS configurations, engagement
conditions, and target phenomena.”
LTG Patrick O’Reilly, Director, MDA
February 25, 2009 Testimony to the House Armed Services Committee, Subcommittee
on Strategic Forces
MDA has implemented a BMDS simulation based acquisition by:
• Revising the M&S Program to focus on BMDS Capability Assessment and
• Refocusing the BMDS Test Program to support data collection for M&S Validation
Approved for Public Release
12- MDA-7017 (24 September 12)

XX/XX/07 3
BMDS Evaluation Strategy is a Direct Result of
Flight Test Program Limitations
Interceptor firing from
Vandenberg Air Force
Base, California
• Lag time to develop new targets that
mimic threat attributes
• Flight Test complexity constraints
- Example: Limited number
of simultaneous engagements
• Flight safety restrictions
• Data collection limitations
• Cost
Flight testing alone can never provide an adequate body of
evidence to fully characterize BMDS performance
Flight tests must be focused on model & simulation validation, the validated
M&S tools can then fully explore the BMDS performance envelope
Approved for Public Release
12- MDA-7017 (24 September 12)

XX/XX/07 4
Balancing Missile Defense Test Venues
Approved for Public Release
12- MDA-7017 (24 September 12)

XX/XX/07 5
A New Paradigm
Flight Test Design Based on M&S Needs
• Critical Engagement
Conditions (CECs)
- Key test points and conditions
where model anchoring data
must be collected in flight tests,
ground tests, exercises, and
real-world operations
- CECs are physics-based
parameters that define specific
conditions under which
engagements must occur
A
L
T
I
T
U
D
E
Interceptor Launch Point
Non-linear
Performance
Boundary
R A N G E
Generic Interceptor Performance Envelope
Note: The primary objectives for flight tests are tied to CECs
Critical Engagement Conditions -- CECs,
not specific threat attributes, are the primary flight test design driver
Approved for Public Release
12- MDA-7017 (24 September 12)
1
Range Safety
Constraint
Stresses
Seeker
Performance
2
3
4
Stresses Sensor
Performance
Stresses
Interceptor
Kinematics
Critical
Engagement
Condition

XX/XX/07 6
BMDS
System-level Verification and Validation
• The Ballistic Missile Defense System (BMDS) is a complex system of
elements, the integration of which enable a robust, layered defense
to defend against a hostile missile in all phases of flight
• Modeling such a system is inherently complex requiring an end-toend
modeling and simulation (M&S) effort and vigorous systemlevel
verification and validation (V&V) of that M&S
• VV&A is critical to understand system capabilities and provide
confidence in M&S that it’s representative of real-world BMDS
behavior
• System-level V&V activities allows verification and validation of
interfaces between Elements and the resulting emergent behaviors
not observed during individual element V&V efforts
Approved for Public Release
12- MDA-7017 (24 September 12)

XX/XX/07 7
Verification, Validation & Accreditation
Essential and Complex
• Validated and Accredited M&S are Required for BMDS
Performance Assessment and Capability Fielding Decisions
• Validated means that the model represents (to a specified level
of detail) physical reality
• Accreditation is the Official Certification that a Model,
Simulation, or Federation of Models and Simulations and
Associated Data are Acceptable for Use for a Specific Purpose.
• Testing to provide referent data is limited
- Each test requires meticulous M&S involvement to ensure the
correct data, in the correct format under the correct conditions
- Thousands of data parameters need to be collected under many
operationally realistic conditions, assessed, and where necessary
implemented in a variety of M&S representations
Approved for Public Release
12- MDA-7017 (24 September 12)

Pre-Mission Test
(PMT)
Flight Test (FT)
XX/XX/07 8
M&S Verification & Validation Process
Post Flight
Reconstruction (primary
M&S validation method)
Digital
(EPFR/SPFR)
DSA-P Validation
HWIL
(EPFR/SPFR)
HWIL System
Validation
CEC/EMEs
M&S Improvement
Simulation Trouble or Incident Reports (STIRs)
BMDS Deficiency Reports (BDRs)
M&S Status Update
System-level V&V
to support…
Technical & Performance
Assessment Events
(TAs / PAs)
Performance Parameter/
Acceptability Criteria
DSA-P
Verification & Validation
Ground Tests (GT)
HWIL System
Verification & Validation
M&S
Element/System
Accreditation
Digital System
Accreditation
HWIL System
Accreditation
Accredited M&S
used for:
- Capability Delivery
- Operational Testing
- BMDS Operational
Performance
Prediction
Data Collected From Flight and Ground Tests is Essential to Successful
Verification, Validation and Accreditation of MDA M&S
Approved for Public Release
12- MDA-7017 (24 September 12)

XX/XX/07 9
PAA Phase I M&S Architectures
DSA-P PAA Phase I Configuration
SSF PAA Phase I Configuration
Approved for Public Release
12- MDA-7017 (24 September 12)
M&S Assessment Tool
Set is maturing through
flight test pre-mission
analysis, post flight
reconstruction, ground
testing and
performance
assessment using
integrated frameworks
and Element
representations
increasing confidence
to validate the PAA
Phase 1 Mission
Thread

100
80
60
40
20
0
XX/XX/07 10
Increased Demand for Modeling & Simulation
Increasing # of Element HWIL Nodes – Increases
Architecture Complexity
FY08 FY09 FY10 FY11 FY12
ET nodes MDSE nodes SSF SIUs
Dramatically Increasing Demand for Modeling and Simulation across MDA is
Empirical Evidence of Success in Adopting Simulation Based Acquisition
Approved for Public Release
12- MDA-7017 (24 September 12)
High Volume
Moderate Complexity
Moderate Volume
High Complexity

XX/XX/07 11
A Framework built on existing
SSF & DSA Architectures to
produce a modular, scale-able,
easily reconfigurable simulation
with both HWIL & Digital
constituents:
- An extensible architecture (e.g.,
SOA) to simplify heterogeneous
interfaces
- A master “clock” to provide
singular event control (“Hollywood
Rule”)
- A library of common software
objects: an Agile Computing
Environment
Objective Simulation Architecture
A Unified Framework
The framework code is not allowed to be modified. Users can extend the
framework (i.e., extensibility), but not modify its code.
OSF
The Next Evolution in MDA Modeling and Simulation
Approved for Public Release
12- MDA-7017 (24 September 12)

XX/XX/07 12
M&S Strategic End State
• Execution of a technical vision toward a single, coordinated M&S
strategy
• VV&A remains essential
• M&S Program ever evolving to meet future BMDS
• Accredited M&S products capable of addressing the needs of
stakeholders
• Comprehensive set of modular, composable, scaleable, and
extensible models that integrate seamlessly with Digital & HWIL
environments
Achieve the MDA Goal of Reducing Agency Costs Using Accredited M&S to
Complement Flight Test and Expand the BMDS Testing Envelope
Approved for Public Release
12- MDA-7017 (24 September 12)