I__. - International Military Testing Association

_.-.- - . . -.-_--_ . .- .~
--...T
TROUBLESHOOTING ASSESSMENT AND ENHANCEMENT (TAE) PROGRAM:
TEST AND EVALUATION RESULTS *
Paper Presented by Dr. Harry B. Conner,
Navy Personnel Research and Development Center, San Diego, CA 921528800
32nd ANNUAL MILITARY TEST& ASSOCIATION CONFERENCE
November f&9,1990, Orange Beach, Alabama
Nauta Rack- (1984) reported on a number of difficulties associated with the U.S. Navy’s ability to
maintain fts weapons systems. He reported the costs of poor performance of maintenance personnel and
recommended areas re uiring investigation if performance of these personnel was to improve. At about the
same time at the Navy 8 ersonnel Research and Development Center (NPRDC), we determined that one of
the difficulties we had encountered in the test and evaluation of an ongoing project (the Enlisted Personnel
Individualized Career System-EPICS) was we had no way of comparing maintenance personnel in the most
important aspect of their erformance: troubleshooting of the hardware system. We realized that we needed
an objective way to eva P uate personnel performance in the skill of troubleshooting. A, literaturessearch
supported the contention that most research and development efforts in this area start with a premtse of a
known expert, journeyman/master, or experienced troubleshooter when in fact these are defined rather than
empirically determined. Therefore, we concluded that efforts to improve maintenance personnel
troubleshooting performance were futile until we could empirically and objectively define how a good
troubleshooter performs.
Aoproach. We addressed this evaluation issue first with a feasibility study (Conner 1988, 1987) followed
by a more structured investigation the Troubleshooting Assessment and Enhancement (TAE) program. The
TAE objective was to design, develop, test, and evaluate a low cost troubleshooting evaluation capability.
The model (Figure 1) we used in our investigation shows that maintenance is just one of a number*of
activities associated with a hardware system. Within the area of maintenance, one can perform preventattve
or corrective maintenance. Within corrective maintenance, one troubleshoots or repairs. Specifically, we
focused on the skill of troubleshooting, which we considered to be a skill of problem solving requiring abstract
conceptualization capabilities.
HARDWARE SYSTEM INTERACTfONS
I I I
CONSTRUCT INSTALL OPERATE MAINTAIN
I 1
PREVENTIVE-- . CORRECTIVE
. .._____.... ^_
MAINTENANCE MAlNTtNANUt
I
I I
+ TROUBLiSHOOTlNG REPAIR
Figure 1. Hardware Activity to Troubleshooting
With 25 subject matter experts, we developed a list of factors to be used to evaluate the proficiency of a
troubleshooting technician in a high tech environment; that is, systems having state-of-the-art electronics and
computers requiring troubleshooting. Next, we sent our initial factors list with definitions (shown in Table 1)
to 1200 operational hi-tech personnel for ranking. The results were then weighted by a jury of experts (on the
system under investigation). Once the factors were weighted, a scoring methodology was developed. Table
2 provides the results of the factor development, weighting, and TAE scoring scheme. Our literature search
caused us to add a tenth factor: redundant checks.
::
3.
4.
2
i:
9.
10.
Rank Factor
Solution.
Cost (Incorrect Solutions).
Time.
Proof Points.
Illogical Approaches.
Invalid Checks.
Out-of-Bounds.
Test Points.
Checks.
Redundant Checks.
TABLE 1. Factor Definitions
-
Definition
Problem is correctly solved; fault is identified.
Number of Lowest Replaceable Units (LRUs) incorrectly identified as faulty.
Total minutes from login to logout taken to find the fault.
Test points that positively identify LRUs as faulty.
Inappropriate equipment selection.
Inappropriate test at appropriate test point.
inappropriate test point was selected.
Total number of valid reference designator tests.
Total number of tests performed at all test points.
Same test performed at same point during the episode. - - - -
� The opinions expressed in this paper are (hose of the author, are not official and da not necessarily reflect Ihe vim+s Of the Navy mPadme”t
372
.
/