Theory of Statistics - George Mason University

7.8 Confidence Sets 537
Monte Carlo simulation would be an appropriate way to study this situation.
The data could be obtained by a process that involves a stopping
rule, and the tests could be performed in a manner that ignores the process.
Whether or not the test is valid could be assessed by evaluating the p-value
under the null hypothesis.
You are asked to explore this issue in Exercise 7.5.
Evidence Supporting Hypotheses
Example 7.4 is a good illustration of the Neyman-Pearson solution to a simple
hypothesis testing problem. We might look at this problem in a slightly different
context, however, as suggested on page 314. This particular example, in
fact, is used in Royall (1997) to show how the data actually provide evidence
that might contradict our decision based on the Neyman-Pearson hypothesis
testing approach.
Example 7.13 Critique of the hypothesis test of Example 7.4
Let’s consider the Bernoulli distribution of Example 7.4 and the two hypotheses
regarding the parameter π, H0 : π = 1/4 and H1 : π = 3/4. The test is
based on x, the total number of 1s in n trials. When n = 30, the Neyman-
Pearson testing procedure at the level α = 0.05 rejects H0 in favor of H1 if
x ≥ 13.
Looking at the problem of choosing H0 or H1 based on the evidence in the
data, however, we might ask what evidence x = 13 provides. The likelihood
ratio in equation 7.16 is
L(P1, x)
= 1/81,
L(P0, x)
which would seem to be rather compelling evidence in favor of H0 over H1.
An additional problem with the test in Example 7.4 is the manner in which
a decision is made if x = 12. In order to achieve an exact size of α = 0.05,
a randomization procedure that does not depend on the evidence of the data
is required. This kind of randomization procedure does not seem to be a
reasonable way to make a statistical decision.
The alternative approach to hypothesis testing involves a comparison of
the evidence from the data in favor of each of the competing hypotheses.
This approach is similar to the use of the Bayes factor discussed in Section
4.5.3.
7.8 Confidence Sets
For statistical confidence sets, the basic problem is to use a random sample
X from an unknown distribution P to determine a random subfamily A(X)
of a given family of distributions P such that
Theory of Statistics c○2000–2013 James E. Gentle