Clinical Trials

❘❙❚■ Chapter 18 | Significance Tests and Confidence IntervalsAlthough the P-value measures the strength of evidence for a difference, which islargely dependent on the sample size, it does not provide the size and direction ofthat difference. Therefore, in a statistical report, P-values should be providedtogether with CIs (described in detail later) for the main outcomes [3].Step 5: Make a statistical inferenceWe are now in a position to interpret the P-value in relation to our data and decidewhether there is sufficient evidence to reject the null hypothesis. Essentially,if P ≤ α, the prespecified significance level, then there is evidence against thenull hypothesis and we accept the alternative hypothesis and say that there is astatistically significant difference. The smaller the P-value, the lower the chance ofobtaining a difference as big as the one observed if the null hypothesis were true,and, therefore, the stronger the evidence against the null hypothesis. Otherwise,if P > α, there is insufficient evidence to reject the null hypothesis, or there is nostatistically significant difference.For our SBP data, since P < 0.05, we can state that there is some evidence to rejectthe null hypothesis of no difference at the 5% significance level, and, therefore,that the mean SBP for the adult male population is statistically significantlydifferent from 129 mm Hg. Furthermore, the actual P-value equals 0.025, whichsuggests that the probability of falsely rejecting the null hypothesis is 1 in 40 if thenull hypothesis is indeed true. On the other hand, Z 0.005= 2.58 > Z = 2.24,calculating P > 0.01. Now we say that there is no evidence to reject the nullhypothesis of no difference if the significance level α is chosen as 0.01.The implementation of the above procedures for hypothesis testing with the SBPdata is summarized in Table 1.Type I (alpha) and Type II (beta) errorsWhen performing a hypothesis test, two types of error can occur. To explain thesetwo types of error, we will use the example of a randomized, double-blind,placebo-controlled clinical trial on a cholesterol-lowering drug ‘A’ in middle-agedmen and women considered to be at high risk for a heart attack. The primaryendpoint is the reduction in the total cholesterol level at 6 months from randomization.The null hypothesis is that there is no difference in mean cholesterol reduction at6 months following randomization between patients receiving drug A (μ 1) andpatients receiving placebo (μ 2) (H 0: μ 1= μ 2); the alternative hypothesis is thatthere is a difference (H a: μ 1≠μ 2). If the null hypothesis is rejected when it is in facttrue, then a Type I error (or false-positive result) occurs. For example, a Type Ierror is made if the trial result suggests that drug A reduced cholesterol levelswhen in fact there is no difference between drug A and placebo. The chosen190