A Technical History of the SEI

Another group inferred the quality of code by examining code samples. The group also analyzed
samples of the documentation for accuracy, readability, and consistency. The other subteams examined
the methods and tools that were to be used in both the development and maintenance environments,
the change-management model, the use of schedulability models, and the processes
that had been used. Several observations were that the FAA maintenance staff in Atlantic City
was using obsolete technology with no relationship to the tools and methods that the contractor
had adopted. In addition, the proposed requirements for interoperability of the tools were unprecedented
and had not been demonstrated to work. There had not previously been a rigorous schedulability
analysis even though this method was now common practice. The subteam also analyzed
the change-management process. While the overall model was sound, the group made recommendations
for setting more effective priorities on needed changes and tracking them.
The general conclusion of the assessment team was that the proposed new architecture was good,
the code was fair, and the documentation was poor. The team provided 14 recommendations, both
technical and non-technical, for the contractor and the FAA to follow if the decision was made to
keep the existing system.
The Consequence: Successful FAA System Upgrade
The assessment team’s 14 recommendations had significant impact because the inspector general
for the DoT was following this assessment closely, as was the General Accounting Office 58
(GAO) [GAO 1999]. After the study was completed, the FAA was directed by the DoT and GAO
to adhere to these 14 points. The recommendations were adopted within the FAA as well as by the
contractor. As a result of the team’s study, the FAA gave the contractor a fixed-price contract to
complete the work. This contract incorporated each of the 14 points of the SEI- Lincoln Laboratory
team. The contractor completed the work, the system was upgraded, and this system is in use
today.
The upgrade process went smoothly enough that all parties involved were able to declare success.
Another positive result of the experience was that the FAA contracted with the SEI for a number
of follow-up projects. This project also represented the first of many SEI independent technical
assessments of non-DoD government acquisition projects. The team documented its approach,
and it was used as a model for a number of these projects. In addition, the team wrote several articles
describing the project to the broader research community and presented these papers at two
technical conferences in 1995, the European Software Engineering and Foundations of Software
Engineering conferences [Brown 1995a, 1995b].
The SEI Contribution
This entire effort was a collaboration between Lincoln Laboratory and the SEI. Lincoln Laboratory
provided the domain knowledge as well as the operational understanding of the FAA. Without
that expertise, the effort would have had a low probability of success. The SEI contributed its
58 At the time this was the General Accounting Office; it is now the Government Accountability Office.
CMU/SEI-2016-SR-027 | SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY 246
Distribution Statement A: Approved for Public Release; Distribution is Unlimited.