Theory of Statistics - George Mason University

Power of a Statistical Test
7.1 Statistical Hypotheses 509
We call the probability of rejecting H0 the power of the test, and denote it by
β, or for the particular test δ(X), by βδ. The power is defined over the full
set of distributions in the union of the hypotheses. For hypotheses concerning
the parameter θ, as in Example 7.1, the power can be represented as a curve
β(θ), as shown in Figure 7.1. We see that the power function of the test, for
any given θ ∈ Θ as
βδ(θ) = Eθ(δ(X)). (7.9)
The power in the case that H1 is true is 1 minus the probability of a
type II error. Thus, minimizing the error in equation (7.7) is equivalent to
maximizing the power within Θ1.
The probability of a type II error is generally a function of the true distribution
of the sample Pθ, and hence so is the power, which we may emphasize
by the notation βδ(Pθ) or βδ(θ). In much of the following, we will assume
that θ ∈ Θ ⊆ IR k ; that is, the statistical inference is “parametric”. This setup
is primarily one of convenience, because most concepts carry over to more
general nonparametric situations. There are some cases, however, when they
do not, as for example, when we speak of continuity wrt θ. We now can focus
on the test under either hypothesis (that is, under either subset of the family
of distributions) in a unified fashion.
Because the power is generally a function of θ, what does maximizing the
power mean? That is, maximize it for what values of θ? Ideally, we would
like a procedure that yields the maximum for all values of θ; that is, one that
is most powerful for all values of θ. We call such a procedure a uniformly
most powerful or UMP test. For a given problem, finding such procedures, or
establishing that they do not exist, will be one of our primary objectives.
In some cases, βδ(θ) may be a continuous function of θ. Such cases may
allow us to apply analytic methods for identifying most powerful tests within a
class of tests satisfying certain desirable restrictions. (We do this on page 521.)
Randomized Tests
We defined a randomized test (page 289) as one whose range is not a.s. {0, 1}.
Because in this definition, a randomized test does not yield a “yes/no” decision
about the hypothesis being tested, a test with a random component is more
useful.
Given a randomized test δ(X) that maps X onto {0, 1} ∪ DR, we can construct
a test with a random component using the rule that if if δ(X) ∈ DR,
then the experiment R is performed with δR(X) chosen so that the overall
probability of a type I error is the desired level. The experiment R is independent
of the random variable about whose distribution the hypothesis applies
to. As a practical matter a U(0, 1) random variable can be used to define
the random experiment. The random variable itself is often simulated on a
computer.
Theory of Statistics c○2000–2013 James E. Gentle