From Algorithms to Z-Scores - matloff - University of California, Davis

224 CHAPTER 11. INTRODUCTION TO SIGNIFICANCE TESTS
to test H0 : θ = c, form the test statistic
Z = θ − c
s.e.( θ)
where s.e.( θ) is the standard error of θ, 2 and proceed as before:
Reject H0 : θ = c at the significance level of α = 0.05 if |Z| ≥ 1.96.
11.3 Example: Network Security
(11.6)
Let’s look at the network security example in Section 10.8.1 again. Here θ = X − Y , and c is
presumably 0 (depending on the goals of Mano et al). From 10.42, the standard error works out to
0.61. So, our test statistic (11.6) is
Z =
X − Y − 0
0.61
= 11.52 − 2.00
0.61
= 15.61 (11.7)
This is definitely larger in absolute value than 1.96, so we reject H0, and conclude that the population
mean round-trip times are different in the wired and wireless cases.
11.4 The Notion of “p-Values”
Recall the coin example in Section 11.1, in which we got 62 heads, i.e. Z = 2.4. Since 2.4 is
considerably larger than 2.4, and our cutoff for rejection was only 1.96, we might say that in some
sense we not only rejected H0, we actually strongly rejected it.
To quantify that notion, we compute something called the observed significance level, more
often called the p-value.
We ask,
We rejected H0 at the 5% level. Clearly, we would have rejected it even at some small—
thus more stringent—levels. What is the smallest such level?
2 See Section 10.5. Or, if we know the exact standard deviation of θ under H0, which was the case in our coin
example above, we could use that, for a better normal approximation.