A Technical History of the SEI

The Consequence: Effective Evaluation of Architecture Designs
Building on the SAAM, the ATAM represents a further evolutionary development in the use of
scenarios to perform architecture evaluation. Major defense contractors, such as Boeing and Raytheon,
now have architecture evaluation teams and architecture evaluation as an element of their
architect certification processes. The U.S. Army has reported that use of the ATAM and other scenario-based
architecture evaluation methods reduced
risk in schedule and cost, improved documentation, and
resulted in a higher quality product [SEI 2010]. Both
General Motors and the Air Force Space Surveillance
Network Model (SSNAM) used the results of SEIconducted
ATAM evaluations to inform crucial decisions
about system-evolution paths.
Many published case studies document positive results
from the use of the ATAM to evaluate software architectures,
including architectures for the U.S. Army’s Warfighter
Information Network-Tactical (WIN-T) system
[Clements 2005]; the Common Avionics Architecture
System (CAAS) for a family of U.S. Army Special Operations
helicopters [Barbacci 2003]; the Wargame 2000
system at the Joint National Integration Center (formerly
known as the Joint National Test Facility [JNTF]) in
Colorado [Jones 2001]; and a reference architecture for
ground-based command-and-control systems [Gallagher
2000].
The SEI completed a study of the impact of the ASSIPsponsored
Quality Attribute Workshops (QAWs) and architecture
evaluations using the ATAM. Ten of the
eleven programs responding to the survey indicated that
the ATAM/QAW produced better results than they traditionally
obtained. Six of the ten indicated that the
ATAM/QAW costs no more than their traditional approaches.
The study also provided evidence of the potential
for the ATAM/QAW having even more impact if
done “proactively.”
The SEI Contribution
The View from Others
The year [2008] also saw a scaling
up of the Army’s interest in learning
and applying the SEI’s software
architecture knowledge
through ASSIP. A concerted effort
conducted through the SEI helped
the Army grow its ranks of software
experts trained in the SEI Architecture
Tradeoff Analysis
Method (ATAM). Army personnel
have taken part in about a dozen
ATAM evaluations to date. The
Army has also seen an added, immediate
benefit from the architecture
training: The PEOs have used
them to reveal software risks early
in projects’ lifetimes.
[SEI 2009, p. 17]
It’s a towering accomplishment to
get everyone’s heads wrapped
around this and get all of these important
issues distilled in this
amount of time.
– General Motors staff
on the ATAM
The development of the ATAM and its ultimate codification as a methodology for dealing with
tradeoffs among quality attributes, was influenced by a 1996 workshop that the SEI conducted.
Participants from the Georgia Institute of Technology, Motorola, IBM T.J. Watson Research, Carnegie
Mellon University, Siemens, Nortel, Rational, AT&T Bell Laboratories, and Performance
Engineering Services met with SEI staff members to discuss and validate an SEI report on best
practices in architecture evaluation [Abowd 1996]. The workshop helped to shape the structure of
the evaluation method that evolved into the ATAM, which includes the Quality Attribute Workshop.
The SEI also published case studies documenting the use of QAWs in source selection for a
CMU/SEI-2016-SR-027 | SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY 259
Distribution Statement A: Approved for Public Release; Distribution is Unlimited.