Understanding Clinical Trial Design - Research Advocacy Network

UNDERSTANDING CLINICAL TRIAL DESIGN: A TUTORIAL FOR RESEARCH ADVOCATES
The calculation of test-statistics is beyond the scope of this tutorial. However,
three factors always influence their values, and hence how likely they are to deviate
from test statistics that would have been computed had the null hypothesis been
true. These are:
1) Sample Size: The larger the sample, the more likely the observed outcomes
reflect their overall populations. The limiting case is when the entire population
is part of the clinical trial. Thus, other factors being equal, the larger the sample
size, the more likely the null hypothesis will be rejected.
2) Variability: The less variability among patients within each group, the more
likely they reflect the overall populations. In trials with low variability, trial
outcome differences between experimental and control arms are likely to be real
(i.e., not due to chance). Thus, other factors being equal, the null hypothesis is
more likely to be rejected in trials with low variability.
3) Outcome Difference: The larger the difference in outcomes between the experimental
and control arms, the more likely there is a true difference, even if it is
actually smaller or larger than observed in the trial. Thus, other factors being
equal, the larger differences between experimental and control arms, the more
likely the null hypothesis will be rejected.
How unlikely would the trial results need to be to reject the null hypothesis? This
rests on researchers’ tolerance for errors. The more tolerant of errors, for example,
in more exploratory work, the more likely the null hypothesis will be rejected.
However, it is never actually known whether or not the null hypothesis is true.
Rather, researchers establish criteria to maintain specific error rates across all trials.
This is what they do when they set α or type I error rates at .5%, and β or type II
error rates at 20%. Figure 9 and the following text explain these potential errors
in more detail. This material is a rather technical and some readers may choose
to skip to the judicial example at the end of this section.
The columns of Figure 9 provide the two possible true states of affairs: either the
null hypothesis (H 0 ) is true or false. The rows give the two possible decisions;
either fail to reject or reject the null hypothesis. The cells show the conclusions
that are drawn—either the experimental and control groups are equivalent or not—
and whether the decision was correct (indicated by ☺), or an error (indicated by
).
Figure 9. Errors in Hypothesis Testing
Truth/Decision H 0 is True H 0 is False
Fail to Reject H 0
Reject H 0
α or type I error
☺
Ex = Control
Ex = Control
Ex = Control
☺ ☺
Ex = Control
19
β or type II error