A Technical History of the SEI

Assurance Cases
The Challenge: Confidence in the Behavior of Performance-Critical
Systems
A concern surrounding DoD systems was the need to shorten the certification process for safety,
system reliability, or security. Traditional software and systems engineering techniques, including
conventional test and evaluation approaches, were unable to provide the justified confidence
needed. Consequently, a methodology to augment testing and evaluation was needed. The DoD
needed to identify which parts are most important and which are less important, thereby enabling
a more economically justified allocation of resources to the important points and issues. When a
system had to be recertified because of a change to the system, the DoD needed to determine what
must be changed, saving costly rework.
A Solution: Assurance Cases
The SEI had long experience in areas related to the DoD certification needs. SEI work in rate
monotonic analysis (RMA) and in Simplex led to a more general interest in performance-critical
systems, which became an area of concentration for the SEI in the mid-1990s. The SEI’s investigation
of performance-critical systems broadened the focus beyond real time, which had been the
focus of RMA. The Simplex architecture allowed for improvements in performance without sacrificing
the need to assure safety. Simplex provided a highly reliable core with higher performance
code around this core area. If the higher performance code began to misbehave in some way, the
system would automatically revert to the core, thereby providing higher performance when possible
along with the assurance of safety in all situations. SEI work in dependable systems upgrades
[Gluch 1997, 1998] was intended to extend this basic Simplex idea to a formal analysis of requirements
and an effort to guarantee particular quality attributes.
Around the same time, the SEI became interested in fault-tolerant computing as it applied to software.
SEI staff members developed a conceptual framework [Heimerdinger 1992], organized and
hosted a series of dependable-software technology exchanges [Weinstock1993], and worked with
the National Institute of Standards and Technology (NIST) to establish a Center for High Integrity
System Software Assurance. As the SEI investigated performance-critical systems, and as systems
of systems became more prevalent, the difficulty of assuring the safety, security, or reliability of
net-centric systems of systems became clear—because of their size, complexity, and continuing
evolution, and because they can exhibit undesired and unanticipated emergent behavior (actions
of a system as a whole that are not simple combinations of the actions of the individual constituents
of the system).
In considering the DoD certification problem, SEI experts focused on assurance cases. An assurance
case provides a means to structure the reasoning that engineers use implicitly to gain confidence
that systems will work as expected. It also becomes a key element in the documentation of
the system and provides a map to more detailed information.
The concept of an assurance case was derived from the safety case, a construct that had been used
successfully in Europe for more than a decade to document safety for nuclear power plants, transportation
systems, automotive systems, and avionics systems. Much like a legal case presented in
a courtroom, an assurance case requires arguments linking evidence with claims of conformance
to the requirements of interest. It includes (1) a claim embodying what we want to show (e.g., the
CMU/SEI-2016-SR-027 | SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY 226
Distribution Statement A: Approved for Public Release; Distribution is Unlimited.