Clinical Trials

❘❙❚■ Chapter 29 | MultiplicityIntroductionThe most simple randomized clinical trial involves the comparison of twotreatments with respect to just one outcome measure. In this case, the observeddata can be evaluated with a statistical significance test where a traditionalthreshold for significance (such as P < 0.05) is chosen as evidence of a truedifference between the two treatments. The problem of multiplicity arises whena clinical trial is used to test several hypotheses simultaneously, rather than justa single test for a single hypothesis based on a single event outcome [1].The key to understanding the problem of multiplicity is that we may considera treatment effect to be statistically significantly superior to the control treatmentwhen indeed the difference arose by chance. This is called a Type I error(see Chapter 18). To elaborate, consider a randomized double-blinded placebocontrolledclinical trial evaluating drug A for reducing systolic blood pressure(SBP) among hypertensive patients. The null hypothesis is that there is nodifference in mean SBP between patients receiving drug A (μ 1) and placebo (μ 2)(H 0: μ 1= μ 2). The alternative hypothesis states there is a significant differencebetween treatments (H a: μ 1≠μ 2).If the null hypothesis is rejected due to a chance significant (false-positive)finding, we say that a Type I error has occurred. For example, a Type I error hasoccurred if we claim that drug A reduces SBP when in fact there is no differencebetween drug A and placebo in the reduction of SBP. The probability ofcommitting a Type I error is known as the level of significance, denoted by α.This is traditionally set at α = 0.05 = 5%, suggesting that this scenario will occurin one out of every 20 studies.If a statistical model is constructed for detecting the difference in a parametersuch as the mean between two identical populations (ie, no true differences),with a certain pre-set statistical limit α, then the probability of a Type I errorwill be increased as further tests are conducted using different samples from thetwo same populations and the same limit α. In other words, the chance of findingat least one significant result (overall Type I error) increases with the number oftests, even though there are no true differences between the comparison groups.For instance, assuming that all tests are independent, the probability of one ofthem being spuriously significant is [1 – (1 – α) n ], where n is the number of tests(see Figure 1).From this graph, we can see that if five such tests are performed, the Type I erroris increased to over 0.20. In other words, if five independent tests are used to testnull hypotheses that are in fact true using the significance limit of 0.05 for each330