Probabilistic Performance Analysis of Fault Diagnosis Schemes

Definition 3.9. The upper boundary between the set W k and the ideal point (0,1) is called
the receiver operating characteristic (roc) for the set of all tests V .
Since the set W k changes with time, the roc is time-varying, as well. Also, since W k is
convex (Fact 3.6), the roc is concave. By Fact 3.8, there is a equivalent convex curve that
separates W k from the point (1,0). However, the term roc only refers to the upper boundary.
Characterizing the ROC
Although it may not be possible to compute the roc for the set of all tests V , the set of
tests whose performance points lie on the roc can be characterized theoretically. For any
α ∈ (0,1], let V α be the set of tests for which P f,k ≤ α, at time k. The set of Neyman–Pearson
tests are defined as
V np = argmax P d,k (V ). (3.15)
V ∈V α
In general, the set V α is too abstract to properly formulate and solve this constrained optimization
problem. However, the following lemma shows that V np is nonempty and explicitly
characterizes one element in V np .
Lemma 3.10 (Neyman–Pearson [71]). The likelihood ratio test with P f,k = α is in V np .
Therefore, the roc is given by the set of likelihood ratio tests (see [61] for details).
In the optimization problem (3.15), the probability of a false alarm is constrained to
be less than some α ∈ (0,1]. However, we can also interpret the roc in terms of the vector
optimization problem
max (−P f,k,P d,k ). (3.16)
V ∈V
Since the objective takes values in [0,1] 2 , it not immediately clear what it means for one
point to be better than another. Clearly, the ideal point (0,1) is the best and points on the
diagonal are of little use. The notion of Pareto optimality provides one way to compare
values of the objective (−P f,k ,P d,k ). We say that a point (P f,k ,P d,k ) = (α,β) is Pareto optimal
if no other test can simultaneously improve both P f,k and P d,k . That is, for any other test
with performance (α ′ ,β ′ ) ≠ (α,β), either α ′ > α or β ′ < β. Hence, the roc can be defined
as the set of Pareto optimal points for the vector optimization problem (3.16). One wellknown
method for generating the set of Pareto optimal points (i.e., the roc) is to solve the
“scalarized” optimization problem
max
V ∈V
−γP f,k + (1 − γ)P d,k (3.17)
for all γ ∈ [0,1] [5, 106]. Since the roc is concave, a lower bound may be computed by
solving (3.17) at a finite collection of points 0 < γ 0 < γ 1 < ··· < γ m < 1 and linearly interpo-
34