Theory of Statistics - George Mason University

510 7 Statistical Hypotheses and Confidence Sets
A test with a random component may be useful for establishing properties
of tests or as counterexamples to some statement about a given test. (We
often use randomized estimators in this way; see Example 5.25.)
Another use of tests with random components is in problems with countable
sample spaces when a critical region within the sample space cannot be
constructed so that the test has a specified size.
While randomized estimators rarely have application in practice, randomized
test procedures can actually be used to increase the power of a conservative
test. Use of a randomized test in this way would not make much sense
in real-world data analysis, but if there are regulatory conditions to satisfy, it
might needed to achieve an exact size.
7.2 Optimal Tests
Testing statistical hypotheses involves making a decision whether or not to
reject a null hypothesis. If the decision is not to reject we may possibly make
a secondary decision as to whether or not to continue collecting data, as we
discuss in Section 7.6. For the moment, we will ignore the sequential testing
problem and address the more basic question of optimality in testing. We first
need a measure or criterion.
A general approach to defining optimality is to define a loss function that
increases in the “badness” of the statistical decision, and to formulate the risk
as the expected value of that loss function within the context of the family
of probability models being considered. Optimal procedures are those that
minimize the risk. The decision-theoretic approach formalizes these concepts.
Decision-Theoretic Approach
The decision space in a testing problem is usually {0, 1}, which corresponds
respectively to not rejecting and rejecting the hypothesis. (We may also allow
for another alternative corresponding to “making no decision”.) As in the
decision-theoretic setup, we seek to minimize the risk:
R(P, δ) = E L(P, δ(X)) . (7.10)
In the case of the 0-1 loss function and the four possibilities, the risk is
just the probability of either type of error.
We want a test procedure that minimizes the risk, but rather than taking
into account the total expected loss in the risk (7.10), we generally prefer to
restrict the probability of a type I error as in inequality (7.6) and then, subject
to that, minimize the probability of a type II error as in equation (7.7), which
is equivalent to maximizing the power under the alternative hypothesis. This
approach is minimizes the risk subject to a restriction that the contribution
to the risk from one type of loss is no greater than a specified amount.
The issue of a uniformly most powerful test is similar to the issue of a
uniformly minimum risk test subject to a restriction.
Theory of Statistics c○2000–2013 James E. Gentle