A Technical History of the SEI

Ada/Real-Time Embedded Systems Testbed
The Challenge: Evaluating Runtime Performance on Embedded Processors
How real-time systems would be programmed in a high-level language like Ada was just one aspect
of Ada adoption for the DoD. Two other important aspects concerned (1) the performance of
the code generated by Ada compilers for the various embedded processors used by the DoD and
(2) the efficiency of the services provided by the Ada runtime environment. The runtime environment
provided services such as process management, storage management, and exception handling
for supporting the execution of Ada programs. Prior to the adoption of Ada, such services
had been provided either by the application programmer or by a small real-time executive.
There was concern both inside and outside the DoD that about whether Ada could support these
real-time needs efficiently. In particular, the semantics of the tasking model and the processing
overhead associated with task interactions and context switching were viewed as impediments to
the real-time performance demanded of mission-critical software. An SEI report describes the issues
and summarizes some of the significant early investigative work of organizations such as the
Ada Runtime Environment Working Group (ARTEWG)—a special interest group established by
the ACM in 1985—and the Evaluation and Validation team of the DoD’s Ada Joint Program Office
[Weiderman 1987a].
A Solution: A Testbed for Real-Time Performance Evaluation
Any assessment of Ada for mission-critical computing on embedded processors would have to
take into account the quality of both the generated code and the runtime execution environment.
The SEI established the Ada Embedded Systems Testbed (AEST) in 1986 to investigate these
questions. The objective was to generate and disseminate quantitative evaluations of a representative
set of vendors’ Ada implementations targeted to various embedded processors. The investigations
used a test suite of Ada programs comprising existing Ada benchmarks, a simulated realtime
application based on a Navy shipboard inertial navigation system (INS), and a new benchmark
created specifically for the project.
Criteria for constructing the testbed included requirements for each tested compiler (for example,
the smallest value of the pre-defined “Duration” type should be less than 100 microseconds) and
its runtime system (for example, the overhead for a context switch should be less than 200 microseconds)
[Weiderman 1987b]. An investigation of existing benchmarks led to the selection of the
University of Michigan Ada benchmarks [Clapp 1986] and the ACM Special Interest Group on
Ada (SIGAda) Performance Issues Working Group (PIWG) benchmarks as the initial test suites
[Donohoe 1987].
The testbed itself was a host-target environment in which a cluster of Digital Equipment Corp.
Microvax II host machines were connected to a set of target single-board computers with processors,
such as a 20 MHz Motorola MC68020, a 16 MHz Intel i80386, and a 15 MHz Fairchild
1750A (with a MIL-STD-1750A instruction set architecture). The Ada cross-compilers for the target
boards came from DDC-I, Systems Designers, Tartan Laboratories, TeleSoft, and Verdix. The
testbed also included a logic analyzer because one of its construction criteria was the requirement
for hardware verification of software timing results. The effort soon evolved beyond the objective
of evaluating Ada and was broadened into the Real-Time Embedded Systems Testbed (REST).
CMU/SEI-2016-SR-027 | SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY 27
Distribution Statement A: Approved for Public Release; Distribution is Unlimited