Probabilistic Performance Analysis of Fault Diagnosis Schemes

3.3.3 Aggregate Measures of Performance
Although the performance metrics {P tn,k }, {P fp,k }, {P fn,k }, and {P tp,k } fully characterize the
time-varying behavior of the fault detection scheme V = (F,δ), it is often useful to aggregate
these probabilities into a single meaningful quality. In this section, we consider two common
aggregate performance measures. These approaches are included to further elucidate
the connection between statistical hypothesis testing and performance analysis for fault
detection schemes.
Probability of Correctness
The probability of correctness of a test V , denoted c k , is defined as the probability that the
decision d k corresponds to the correct hypothesis. More precisely, for each time k,
c k := P tn,k + P tp,k = (1 − P f,k )Q 0,k + P d,k Q 1,k .
Equivalently, one may consider the probability e k := 1−c k , which is known as the probability
of error [61].
Bayesian Risk
To generalize the concept of accuracy, we utilize the concepts of loss and risk used in
hypothesis testing [60] and general statistical decision theory [4, 22]. Fix a time k. In general,
a loss function L k : Θ × D → R is a nonnegative bounded function that quantifies the loss
L k (ϑ k ,d k ) incurred by deciding d k when ϑ k is the true state of affairs. Since the parameter
space is partitioned as Θ = Θ 0 ∪ Θ 1 and the set of decisions is D = {0,1}, a loss function for
the fault detection problem can be expressed as a matrix L k ∈ R 2×2 with nonnegative entries.
The value L k (i , j ) can be interpreted as the loss incurred by deciding d k = j “averaged” over
all ϑ k ∈ Θ i .
The loss matrices {L k } k≥0 provide a subjective way to quantify the importance of making
the correct decision in each possible case. The Bayesian risk R k (Q,V ) is defined to be the
expected loss incurred by the test V at time k, given that the parameter {θ k } is distributed
according to Q k = {Q 0,k ,Q 1,k }. More precisely, for each time k,
( )
R k (Q,V ) := E L(θ k ,d k )
In terms of the performance metrics, the risk is
=
1∑
i=0 j =0
1∑
L k (i , j )P(D j,k ∩ H i ,k ).
R k (Q,V ) = L(0,0)P tn + L(1,0)P fn + L(0,1)P fp + L(1,1)P tp
(
) (
)
= L(0,0)Q 0 + L(1,0)Q 1 + L(0,1) − L(0,0) P f Q 0 + L(1,1) − L(1,0) P d Q 1 ,
(3.13)
29