2003 IMTA Proceedings - International Military Testing Association

4
A major consideration is the impact of using face-to-face groups on ratings and
evaluations. Pill (1970) has said that this may dilute the opinions of the real expert. This
seems a strange objection as the SME’s selected are the real experts. However, if what is
meant is that one of the groups may have more expertise in a small, specific area that is being
worked on, then the reality (based on conducting 7 or so of these) is that the other SMEs
recognize, welcome and use that expertise –as their goal is the best product possible.
Another objection is that the group dynamics may force ratings and analyses towards
a mean or middle ground that does not fully reflect all the SMEs views. There are 2 answers
to this: the first is that “real” SMEs will not allow that to happen for personal reasons and
pride. They will not “go along to get along.” The second is that the instruction in group
work, the facilitator and an iterative methodology used in accomplishing the sub-objectives
are all structures in the SGDP process designed to ensure that this does not happen.
CONCLUSION
While the use of the SGDP technique has been suggested for building operability
testing scenarios and the evaluative criteria for them, it seems that its use may be extended.
In developing the content for any usability test, to include requirements, procedures,
scenarios and evaluation, a variant of the SGDP can be used. As stated previously, the goal
of conducting any operability testing on a system is to ascertain the operational readiness of
the system, i.e., the operational effectiveness and the operational suitability. The
identification and refinement of system-critical operational requirements are eminently suited
to being accomplished via the SGDP.
REFERENCES
Dumas, J.S., Redish, J.C., 1999, “A Practical Guide to Usability Testing”. Intellect: Portland,
OR.
Gibb, G.M., Lofaro, R.J., et al., 1991,“The Development of Enhanced Screening Techniques
for the selection of Air Traffic Controllers” Proceedings of the annual Air Traffic Controller
Association (ATCA) Symposium September 1991
Judd, R.C. (1972). “Use of Delphi Methods in higher Education”, Technological Forecasting
and Social change, 4, 176-196
Lofaro, R.J., 1998, “Identifying and Developing Managerial and Technical Competencies:
Differing Methods and Differing Results” (Proceedings of 42 nd annual Human Factors and
Ergonomics Society.
Lofaro, R.J., Gibb, G.M., and Garland, D., 1994, “A Protocol for Selecting Airline Passenger
Baggage Screeners”) DOT/FAA/CT-94/110. National Technical Information Service:
Springfield, VA.
Lofaro, R.J. and Intano, G.P., 1989. “Exploratory Research and Development: Army Aviator
Candidate Classification By Specific Helicopter” Proceedings of 5th International
Symposium of Aviation Psychologists, R.S. Jensen (ed.).
Meister, D (1985), Behavioral Analysis and Measurement Methods. New York: Wiley .
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003