2DkcTXceO

254 Conditioning is the issueThe practical importance of conditioning arises because, when it is notdone in certain scenarios, the results can be very detrimental to science. Unfortunately,this is the case for many of the most commonly used statisticalprocedures, as will be discussed.This chapter is a brief tour of old and new examples that most influencedme over the years concerning the need to appropriately condition. The newones include performing a sequence of tests, as is now common in clinical trialsand is being done badly, and an example involving a type of false discovery rate.23.2 Cox example and a pedagogical exampleAs this is more of an account of my own experiences with conditioning, I havenot tried to track down when the notion first arose. Pierre Simon de Laplacelikely understood the issue, as he spent much of his career as a Bayesianin dealing with applied problems and then, later in life, also developed frequentistinference. Clearly Ronald Fisher and Harold Jeffreys knew all aboutconditioning early on. My first introduction to conditioning was the exampleof Cox (1958).AvariantoftheCoxexample:Every day an employee enters a lab toperform assays, and is assigned an unbiased instrument to perform the assays.Half of the available instruments are new and have a small variance of 1, whilethe other half are old and have a variance of 3. The employee is assigned eachtype with probability 1/2, and knows whether the instrument is old or new.Conditional inference: For each assay, report variance 1 or 3, depending onwhether a new or an old instrument is being used.Unconditional inference: The overall variance of the assays is .5×1+.5×3 = 2,so report a variance of 2 always.It seems silly to do the unconditional inference here, especially when notingthat the conditional inference is also fully frequentist; in the latter, one is justchoosing different subset of events over which to do a long run average.The Cox example contains the essence of conditioning, but tends to bedismissed because of the issue of “global frequentism.” The completely purefrequentist position is that one’s entire life is a huge experiment, and so thecorrect frequentist average is over all possibilities in all situations involvinguncertainty that one encounters in life. As this is clearly impossible, frequentistshave historically chosen to condition on the experiment actually beingconducted before applying frequentist logic; then Cox’s example would seemirrelevant. However, the virtually identical issue can arise within an experiment,as demonstrated in the following example, first appearing in Berger andWolpert (1984).