Theory of Statistics - George Mason University

3.3 The Decision Theory Approach to Statistical Inference 255
uct of the vector with itself). If the expectation exists, least squares
yields unbiasedness.
Another example of this approach is least absolute values (LAV) estimation,
in which the L1 norm of the vector of residuals is minimized.
This yields median-unbiasedness.
• fit an empirical probability distribution
This approach is somewhat similar to fitting an ECDF, but in the case
of PDFs, the criterion of closeness of the fit must be based on regions of
nonzero probability; that is, it can be based on divergence measures.
• define and use a loss function
(This is an approach based on “decision theory”, which we introduce formally
in Section 3.3. The specific types of estimators that result from this
approach are the subjects of several later chapters.)
The loss function increases the more the estimator differs from the estimand,
and then estimate g(θ) so as to minimize the expected value of the
loss function (that is, the “risk”) at points of interest in the parameter
space.
– require unbiasedness and minimize the variance at all points in the
parameter space (this is UMVU estimation, which we discuss more
fully in Section 5.1)
– require equivariance and minimize the risk at all points in the parameter
space (this is MRE or MRI estimation, which we discuss more fully
in Section 3.4)
– minimize the maximum risk over the full parameter space
– define an a priori averaging function for the parameter, use the observed
data to update the averaging function and minimize the risk defined
by the updated averaging function.
3.3 The Decision Theory Approach to Statistical
Inference
3.3.1 Decisions, Losses, Risks, and Optimal Actions
In the decision-theoretic approach to statistical inference, we call the inference
a decision or an action, and we identify a cost or loss that depends on the
decision and the true (but unknown) state of nature modeled by P ∈ P.
(Instead of loss, we could use its opposite, which is called utility.)
Our objective is to choose an action that minimizes the expected loss, or
conversely maximizes the expected utility.
We call the set of allowable actions or decisions the action space or decision
space, and we denote it as A. We base the inference on the random variable
X; hence, the decision is a mapping from X, the range of X, to A.
In estimation problems, the action space may be a set of real numbers
corresponding to a parameter space. In tests of statistical hypotheses, we may
Theory of Statistics c○2000–2013 James E. Gentle