human factors

15-18 Handbook of Aviation Human Factors
of operators with relevant experience results in fewer false starts, better insight in how and why the
mission is performed, and a great savings in time, as well as money, in the latter steps of the process.
By getting the operator involved from the beginning, the costly problem of making design changes
further down the road is avoided.
The dividing line between problem definition and solution development is often vague. Specific
designs may affect task sequencing during the mission profile. This change in sequencing can reveal
workload problems within the crew station. Because of this overtasking, the operator may shed tasks,
which in turn alter the mission profile. Once the profile has changed, the designs may affect the tasks in
a different way, and thus, the cycle continues. The design process is indeed an iterative process.
15.2.1.1.4 Prototype Evaluation, Simulation Evaluation/Validation, Flight Test
The last three steps are interdependent and very critical to the successful completion of an effective
and proven crew station design. These three steps all work synergistically to “prove the solution.”
Prototype evaluation marks the initial introduction of the implemented design concepts to the user.
Although the users should be involved in the preliminary design step, the actual implementation into
a prototype design will show the design in a whole new light. The design concepts are evaluated in a
limited context, and suggestions are made by the user as to which designs should move forward to
simulation. This step weeds out unfeasible design concepts. Human-in-the-loop simulation evaluation
provides a more realistic and robust testing of the design concepts. In simulation evaluation, it is
recommended that the new design concept be compared to an existing design in order to measure the
“goodness” of the design concept. This step provides the final recommendation of a design concept
for flight test.
Traditionally, this process involved human-in-the-loop simulations, or virtual simulation as they
are referred to today. At present, constructive simulation, which involves the use of models in simulated
environments, is becoming a required part of the evaluation process as a low-cost alternative
to conducting trade studies. Modeling specific systems, such as structures, engines, sensors, etc., for
use in constructive simulation has been very successful (Aviation Week and Space Technology, 2003).
However, one of the current challenges is modeling human behavior. Certainly, to determine the benefits
of different technologies in this step of the design process, the simulation must not only model the
technology, but also how the operator interacts with it. The Combat Automation Requirements Testbed
(CART) program is developing an architecture that allows human behavior/performance models to
interface with various constructive simulation environments to determine the “goodness” of various
cockpit designs and how the operator interfaces with them.
CART has been used to integrate such models successfully (Martin, Barbato, & Doyal, 2004).
In one example, CART was used to model human tasks performed during an air-to-ground segment in
a strike-fighter mission using a human performance model integrated with the Joint Integrated Mission
Model aircraft model. Once the integrated model was run, results from the constructive simulation
were compared with pilot performance from a virtual simulation in which real pilots performed the
same tasks as the model. The human performance model was shown to predict the pilot performance
with fairly high accuracy (correlation of 0.78 between the model-dependent measures and the pilotdependent
measures) (Brett et al., 2002). Once the human performance models are validated, using
constructive simulation prior to virtual simulation can save time and money by providing a quick way
of thoroughly testing design concepts and advancing only the most promising one(s) to virtual simulation
studies.
Flight testing often involves only one design to be tested in operational use; however, in the case of
the F-16, F-22, and the F-35 JSF, two prototypes were involved in a “fly-off.” For the purpose of this
discussion, these final steps are combined to provide “Solution Evaluation.” Once again, there may not
be a clear break between the solution evaluation and the solution definition step. It has been observed
that most designers design, evaluate, redesign, etc., as they go. The transition from solution definition
to solution evaluation occurs when formal, total-mission, total-system, human-in-the-loop evaluations