Theory of Statistics - George Mason University

740 0 Statistical Mathematics
The measure is based on a convex function φ of a term similar to the “odds”.
Definition 0.1.46 (φ-divergence)
Let P be absolutely continuous with respect to Q and φ is a convex function,
dP
d(P, Q) = φ dQ, (0.1.81)
dQ
if it exists, is called the φ-divergence from Q to P.
IR
The φ-divergence is also called the f-divergence.
The φ-divergence is in general not a metric because it is not symmetric.
One function is taken as the base from which the other function is measured.
The expression often has a more familiar form if both P and Q are dominated
by Lebesgue measure and we write p = dP and q = dQ. The Hellinger distance
given in equation (0.1.80) is a φ-divergence that is a metric. The Matusita
distance is the square root of a φ-divergence with φ(t) = ( √ t − 1) 2 .
Another specific instance of φ-divergence is the Kullback-Leibler measure.
Definition 0.1.47 (Kullback-Leibler measure)
Let P be absolutely continuous with respect to Q and p = dP and q = dq.
Then
p(x)
dx. (0.1.82)
q(x)
p(x)log
IR
is called the Kullback-Leibler measure of the difference of p and q.
The Kullback-Leibler measure is not a metric.
Various forms of φ-divergence are used in goodness-of-fit analyses. The
Pearson chi-squared discrepancy measure, for example, has φ(t) = (t − 1) 2 :
IR
(q(x) − p(x)) 2
dx. (0.1.83)
q(x)
See the discussion beginning on page 593 for other applications in which two
probability distributions are compared.
0.1.9.6 Convergence of Functions
We have defined almost everywhere convergence of measurable functions in
general measure spaces (see page 718). We will now define two additional types
of convergence of measurable functions in normed linear measure spaces. Here
we will restrict our attention to real-valued functions over real domains, but
the ideas are more general.
The first is convergence in Lp.
Theory of Statistics c○2000–2013 James E. Gentle