pdf download - Software and Computer Technology - TU Delft

5.7 Results of the Case Study
Diagnosing the Beam Propeller Movement
of the Frontal Stand
• C3: 12 (POSITION_ERROR = true and e_pos = false)
• C4: 1 (POSVAL_ERROR = true)
• Unknown fault category: 1 (error is status).
Table 5.12 shows the entropy gain after weighting for each fault category. For example, the
entropy gain of the sensor set-up [I_mvr, I_to_motor, I_from_motor] is calculated by:
H gain =
16 ∗ 0.2945 + 7 ∗ 0.3729 + 12 ∗ 0.2711 + 1 ∗ 0.1883
36
= 0.2990 (5.6)
The conclusion is that from the considered sensor set-ups, [I_mvr, I_to_motor, I_from_motor]
has the highest entropy gain, thus leads to the highest increase in diagnostic accuracy. Note that the
decision to choose a particular sensor set-up also depends on the costs of adding those specific sensors.
An expert would be unable to order a list of sensor set-ups with respect to higher diagnostic
performance. For example, it is very likely that an expert expects higher improve of accuracy from
the sensor set-up [I_from_motor, Vact] than from sensor set-up [I_mvr, I_set], while this is the
other way around. Ordering a list of sensor set-ups is especially complicating if one likes to consider
all additional sensor set-ups that are possible.
5.7 Results of the Case Study
This section discusses why it is not possible to present the experimental results of the case study.
The reason for this is as follows. The experimental methodology requires that for each fault scenario
it is known what is broken. Unfortunately, for fault scenario’s S1 till S11 and S17 till S39 this is not
possible 4 . These fault scenarios are extracted from the remote monitoring tools, and it is impossible
to determine what components were really broken for those cases. Thus, it is impossible to calculate
the accuracy of the MBD implementations, based on real-life fault scenarios.
In order to still be able to calculate the accuracy of the MBD implementations, 5 faults are injected
in the system in a test situation. This resulted in 5 error messages. Table 5.13 shows the
observations, adjudged broken components, and the outcome of MBD-1 (or MBD-2), for these 5
fault scenarios; S12 till S16. A value of 1 in column A of Table 5.13 means the fault scenario is accurately
diagnosed. The vector that was used to denote the observations is (e_pos, control_speed,
POSVAL_ERROR, POSITION_ERROR, SPEED_ERROR, CURRENT_ERROR).
Using Formula 5.1, the accuracy of the MBD-1 and MBD-2 implementations over these 5 fault
scenarios is calculated:
A MBD−1 = A MBD−2 = 3 = 60% (5.7)
5
Of course, this result should be looked at with care. Only 5 fault scenarios could be used, and
these did not occur in a real case setting. If the set of fault scenarios would be representative for a
large group of real-life fault scenarios, an accuracy of 60% could be interpreted as that the MBD
engine produces the actual diagnosis as the first item in the list for 60% of the fault scenarios. For
the other 40% of the fault scenarios, the actual broken component is in the list, but not as the first
item. For example, the outcome of the LYDIA diagnostic engine for fault scenario S12 lists the
actual broken component as the fourth item. And the outcome of the LYDIA diagnostic engine for
fault scenario S16 lists the actual candidate as the third item.
4 Appendix B.4 lists all fault scenarios.
66