Probabilistic Performance Analysis of Fault Diagnosis Schemes

where the subscript k has been omitted for the sake of clarity.
Example 3.3 (0-1 Loss). Suppose that the loss matrix
[ ]
0 1
L =
1 0
is used for all time. This is typically referred to as “0-1 loss” in the literature [4, 61]. By equation
(3.13), the corresponding Bayesian risk of a test V at time k is
R k (Q k ,V ) = P fp,k + P fn,k
= P f,k Q 0,k + (1 − P d,k )Q 1,k
= 1 − c k .
Thus, placing an upper bound on the 0-1 risk R k (Q k ,V ) is equivalent to placing a lower bound
on the probability of correctness c k .
3.4 Characterizing the Range of Achievable Performance
In Section 3.3, the performance of a test was given in terms of the probabilities {P f,k }
and {P d,k }. In this section, we consider the complementary problem of determining what
performance values (P f,k ,P d,k ) ∈ [0,1] 2 are achievable by some test. Again, we draw on the
tools of statistical hypothesis testing to address this issue. Namely, we use the Neyman–
Pearson Lemma [71] and the receiver operating characteristic (roc) [57] to characterize the
limits of achievable performance. To facilitate our discussion, we first introduce the concept
of a randomized test.
3.4.1 Randomized Tests
Up to this point, we have focused our attention on tests V = (F,δ), where both the residual
generator F and the decision function δ are deterministic. However, it is possible to design
and implement tests that are nondeterministic. In this section, we introduce nondeterministic
or randomized tests and use them to characterize the set of achievable performance
points.
Definition 3.4. A hypothesis test V is said to be a randomized test if, for a given realization
of the test statistic (û 0:k , ŷ 0:k ), the decision d k = V (û 0:k , ŷ 0:k ) is a random variable.
Define V to be set of all deterministic and randomized hypothesis tests, and define W k
to be the set of all performance points (α,β) ∈ [0,1] 2 that are achieved by some test V ∈ V ,
at time k. The following example shows how to derive randomized tests from the class of
30