I__. - International Military Testing Association

TABLE 2. Ranking, Weighting, and Scoring for Troubleshooting Evaluation Factors
Rank Factor Weight Scoring Scale
(Max Points)
Scoring
(Per event)
1
2
3
4
Solution
Cost (Incorrect Sol)
Time
Proof Points
42.78
13.13
11.80
9.88
‘EZ
20:62
17.23
-100 For Fail to find
-0.5 X ea NFR LRU
-0.5 X ea Minute
- % X ea Proof Pt missed
i
Illogical
Invalid
Approach
Checks
6.87
4.68
12.01
8.18
-6.0 X ea Illogical App
-0.8 X ea Invalid Check
5
Out of Sounds
Tests Points
4.00
3.21
8.99
5.61
-0.6 X ea Out of Sounds
4.5 X # of Tests
Q
10
Checks
Redundant Checks
3.08
tbd
5.38
tbd
-0.5 X # of Checks
to be analyzed
Scoring is designed to discriminate between levels of troubleshooting proficiency: failure to solve the
roblem results in a score of 0, while solving the problem results in a score of 100. There is no partial score
Por factor 1. Ability to discriminate between levels of troubleshooting proficiency is in scoring of the remaining
factors. Wei hts for the factors were converted into a scale equaling 100 points. The final score for each
subject equa Bs
100 points minus the sum of points lost for each factor. The minimum score is 0: that is, no
ne atfve scores. The scoring criteria for each factor, also shown in Table 2, are the wei hts that were used
In tfe TAE epfsodes to evaluate and diagnose troubleshooting proficiency levels. The cost9actor
was changed
to incorrect solutions to more accurately describe the actual behavior.
Once we had determined factors and scoring scheme, we selected and constructed practical
troubleshooting episodes that provided a valid representation tionof the hardware system being used in the
study. Our hardware system was the U.S. Navy s communications system, the Navy Modular Automated
Communications System/Satellite Communications (NAVMACS/SATCOM). To construct TAE
troubleshooting episodes, we focused on the fault diagnosis/problem solving behaviors (Table 3) that military
schools have identified in their six step troubleshooting process (Conner 1986,1987).
\
TABLE 3. Six Step Troubleshooting Process
1. Symptom Recognition
2. Symptom Elaboration
3. Probable Faulty Functions
4. Localizing Faulty Function(s)
5. Isolating Faulty Circuit
6. Failure Analysis.
Although the design and delivery of the troubleshooting episodes did not require a computer, the amount
of data made it obvious that the only efficient and cost effective approach would be utilization of microcomputer
delivery and data gathering. Also, to keep developmental and hardware costs down, we limited ourselves to
using off-the-shelf technology. We also reduced the “troubleshooting universe” of the episodes so that a
standard microcomputer memory could handle data.
The model developed for the troubleshooting actlviiy on a iven piece of hardware (shown in Fi ure 2)
9
provides a TAE Factors Model for “System Troubleshooting.” he modd works as follows: Once a8ystem is determined to be inoperative, the fault symptoms reduce the universe of type and location of tests to be
made to a reasonable spectrum for further Investigation; that is the symptoms bound the problem and
establishes what is in or out of bounds. This bounding of the problem reduces the number of tests in the
spectrum to reasonable number and limits the amount of computer memory necessary. We called the “in
bounds” checks that are not logical for the fault symptoms “illogrcal approach.” For a given set of symptoms
for a given fault, there is an optimum troubleshooting path to determine the problem. To prove a component,
or unit, is bad a number of tests must be performed; this requires testing of the “proof points.”
. .
IroubleshootlW
-!+DrK~
Jrcub @shoctlna
f Faull 0 llbgi~l Approach
0 Optimum Path 0 Out Of Bounds
0 Proof Points 0 In Bounds
Figure 2. TAE Factors Model
373