A Technical History of the SEI

system is safe); (2) evidence supporting the claim (e.g., a hazard analysis)—evidence can take on
many forms, including test results, formal analyses, simulation results, fault-tree analyses, hazard
analyses, modeling, and inspections, and (3) an argument explaining how the evidence is linked to
the claim.
It is important that an assurance case be reviewable, which means that having a single claim (“The
system does what it’s supposed to do”) and a single complex argument that links myriad evidence
to the claim are not appropriate. Instead of taking such a large step, the claim is typically broken
into subclaims, each of which can potentially be broken into yet another level of subclaims until
the step to the actual evidence that supports that subclaim is almost obvious.
SEI work in assurance cases was initially funded by the NASA High Dependability Computing
Project, beginning in 2002. On that project, the SEI worked with researchers at Carnegie Mellon
University to introduce advanced thinking into NASA for use with various space projects, including
the Mars Lander and the NASA Mission Data System (MDS), which had an unusual architecture
that raised concerns about reliability. The SEI adapted ideas presented in a PhD thesis by
Kelly at the University of York [Kelly 1998].
The Consequence: Assurance Cases Used in Practice
Application of assurance cases for certification has not been fully implemented yet, but it is an
idea that many are considering and discussing. For example, NASA developed, with SEI assistance,
an assurance case practice. For its Constellation project, the SEI contributed to NASA’s
safety requirements document, which specified the use of an assurance case to demonstrate safety.
Although the assurance case requirement did not survive final review by NASA and its contractors,
and the Constellation project subsequently was cancelled, the idea of assurance cases was
distributed within NASA, leading to some research projects on assurance cases that continue to
this day.
In 2006 the SEI was approached by the U.S. Food and Drug Administration (FDA). As a result of
a number of safety incidents with infusion pumps, the FDA wished to improve the thoroughness
of its review process and to improve the engineering done by manufacturers to assure safety. SEI
work on the use of assurance cases in the development of medical devices [Weinstock 2009] led
directly to the FDA’s issuing draft guidance to manufacturers recommending the use of assurance
cases and providing guidance for their use. As a result, infusion pump manufacturers are beginning
to make use of assurance cases. The FDA is the only official agency of the U.S. government
that has formally mandated the use of assurance cases to date.
The SEI Contribution
The idea that a structured argument is better than an unstructured argument is prevalent in Great
Britain, where the Ministry of Defense (MoD) has for a decade or more required that an assurance-case
kind of structure be presented for certain types of MoD systems. Subsequent to the start
of the SEI’s work in this area, the importance of assurance case concepts was recognized by the
National Research Council in its report “Software for Dependable Systems: Sufficient Evidence?”
[Jackson 2007].
The SEI has been instrumental in developing the assurance case from the existing European safety
case concept, and showing how the cases can be used in various areas, such as aerospace and
CMU/SEI-2016-SR-027 | SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY 227
Distribution Statement A: Approved for Public Release; Distribution is Unlimited.