Statistical Methods in Medical Research 4ed

294 Analysing non-normal data
probability to either element of a pair, with the other element being given
the other label. Moreover, this can be repeated independently for each pair.
In practice, this means that all possible changes of sign can be applied to
each within-pair difference, giving 2n possible permutations when there are n
pairs.
The idea behind permutation tests underlies the justification of several wellknown
tests. This has just been demonstrated for the Wilcoxon rank sum test
and it is also the case for the exact test for 2 2 tables (see §4.5). In the latter
case, if there are two binary samples of sizes r1 and r2 then, in general, several
permutations of the sample labels will give rise to the same table. The number of
such permutations, divided by … r1‡r2†,
will be the probability of obtaining that
r1
table under the null hypothesis; this approach gives a combinatorial argument
for the justification of (4.35). Fisher (1966, para. 21) also used permutation
arguments to justify the use of t tests under certain circumstances, even when
the usual assumption of normality might be inappropriate. The same ideas
underlie randomization tests, which are tests justified by the actual randomization
of treatments to experimental units.
In addition to providing theoretical grounds to justify certain types of tests,
the ideas behind permutation tests can be used directly in applications. They may
be particularly helpful if there is doubt about the validity of other types of tests
or when the calculation of quantities, such as standard errors, needed for other
approaches, is intractable.
Example 10.8
Suppose it was decided to assess whether the standard deviation of weight gain was the
same in the high- and low-protein groups of Example 4 4. The sample standard deviations
are 21 39 g in the high-protein group and 20 62 g in the low-protein group, giving a ratio
of 1 037. A straightforward approach would be to apply the variance ratio test from §5 1
to 1 0372 ˆ 1 076, with degrees of freedom 11 and 6. This gives a two-sided significance
level of 0 979. However, as mentioned in §5 1, the variance ratio test is sensitive to the
assumption that the gains in weight follow a normal distribution and an alternative which
makes no such assumption is a permutation test.
Table 10.5 shows the weight changes and the second column of the table indicates the
type of diet fed to each rat. If the null hypothesis that the standard deviations are equal is
true then the observed ratio, namely 1 037, will be typical of the ratio of the standard
deviations for the groups labelled L and H for any permutation of the column of Ls and
Hs. With 12 observations, in one group and seven in the other there are 19
12 ˆ 50 388
permutations and the ratio of the standard deviations can be computed for each of these;
the results are shown in the histogram in Fig. 10.1. As deviations from the null hypothesis
in either direction are of equal interest, a value should be considered `more extreme' than
the observed ratio of 1 037 if it exceeds 1 037 or if it is less than 1 037 1 ˆ 0 964. There are
46 669 such values, so the permutation P value is 46 669=50 388 ˆ 0 926, which is very
similar to that obtained from the variance ratio test.