2DkcTXceO

262 Conditioning is the issueTABLE 23.1Values of the lower bound α(s) in (23.4) for various values of p.p .2 .1 .05 .01 .005 .001 .0001 .00001α(s) .465 .385 .289 .111 .067 .0184 .0025 .0003123.5.4 Testing a sequence of hypothesesIt is common in clinical trials to test multiple endpoints but to do so sequentially,only considering the next hypothesis if the previous hypothesis was arejection of the null. For instance, the primary endpoint for a drug might beweight reduction, with the secondary endpoint being reduction in an allergicreaction. (Typically, these will be more biologically related endpoints butthe point here is better made when the endpoints have little to do with eachother.) Denote the primary endpoint (null hypothesis) by H0, 1 and the statisticalanalysis must first test this hypothesis. If the hypothesis is not rejectedat level α, the analysis stops — i.e., no further hypotheses can be considered.However, if the hypothesis is rejected, one can go on and consider the secondaryendpoint, defined by null hypothesis H0.Supposethishypothesisis2also rejected at level α.Surprisingly, the overall probability of Type I error (rejecting at least onetrue null hypothesis) for this procedure is still just α — see, e.g., Hsu andBerger (1999) — even though there is the possibility of rejecting two separatehypotheses. It appears that the second test comes “for free,” with rejectionallowing one to claim two discoveries for the price of one. This actually seemstoo remarkable; how can we be as confident that both rejections are correctas we are that just the first rejection is correct?If this latter intuition is not clear, note that one does not need to stopafter two hypotheses. If the second has rejected, one can test H0 3 and, if thatis rejected at level α, one can go on to test a fourth hypothesis H0,etc.Suppose4one follows this procedure and has rejected H0,...,H 1 0 10 .Itisstilltruethatthe probability of Type I error for the procedure — i.e., the probability thatthe procedure will result in an erroneous rejection — is just α. Butitseemsridiculous to think that there is only probability α that at least one of the 10rejections is incorrect. (Or imagine a million rejections in a row, if you do notfind the argument for 10 convincing.)The problem here is in the use of the unconditional Type I error to judgeaccuracy. Before starting the sequence of tests, the probability that the procedureyields at least one incorrect rejection is indeed, α, butthesituationchanges dramatically as we start down the path of rejections. The simplestway to see this is to view the situation from the Bayesian perspective. Considerthe situation in which all the hypotheses can be viewed as aprioriindependent(i.e., knowing that one is true or false does not affect perceptions of the