Theory of Statistics - George Mason University

846 Appendix B. Useful Inequalities in Probability
Corollary B.5.1.5 (Hammersley-Chapman-Robbins inequality)
Let X be a random variable in IR d with PDF p(x; θ) and let Eθ(T(X) = g(θ).
Let µ be a fixed measure on X ⊆ IR d such that p(x; θ) ≪ µ. Now define S(θ)
such that
p(x; θ) > 0 a.e. x ∈ S(θ)
p(x; θ) = 0 a.e. x /∈ S(θ).
Then
V(T(X)) ≥ sup
t∋S(θ)⊇S(θ+t)
(g(θ + t) − g(θ)) 2
p(X;θ+t) . (B.24)
2
Eθ
p(X;θ)
Proof. This inequality follows from the covariance inequality, by first considering
the case for an arbitrary t such that g(θ + t) = g(θ).
Corollary B.5.1.6 (Kshirsagar inequality)
Corollary B.5.1.7 (information inequality)
Subject to some “regularity conditions” (see Section 2.3), if X has PDF
p(x; θ), then
V(T(X)) ≥
Eθ
2 ∂E(T(X))
∂θ
(B.25)
2
∂ log p(X;θ)
The denominator of the quantity on the right side of the inequality is
called the Fisher information, or just the information. Notice the similarity
of this inequality to the Hammersley-Chapman-Robbins inequality, although
the information inequality requires more conditions.
Under the regularity conditions, which basically allow the interchange of
integration and differentiation, the information inequality follows immediately
from the covariance inequality.
We consider the multivariate form of this inequality to Section 3.1.3. Our
main interest will be in its application in unbiased estimation, in Section 5.1.
If T(X) is an unbiased estimator of a differentiable function g(θ), the right
side of the inequality together with derivatives of g(θ) forms the Cramér-Rao
lower bound, inequality (3.39), and the Bhattacharyya lower bound, inequality
(5.29).
Theorem B.5.2 (Minkowski’s inequality)
For 1 ≤ p,
(E(|X + Y | p )) 1/p ≤ (E(|X| p )) 1/p + (E(|Y | p )) 1/p
∂θ
(B.26)
This is a triangle inequality for Lp norms and related functions.
Proof. First, observe the truth for p = 1 using the triangle inequality for
the absolute value, |x + y| ≤ |x| + |y|, giving E(|X + Y |) ≤ E(|X|) + E(|Y |).
Theory of Statistics c○2000–2013 James E. Gentle