statisticalrethinkin..

1.2. WRECKING PRAGUE 21e dominant reaction from the physics community was not “Einstein was wrong!” butinstead “How did the team mess up the measurement?” e team that made the measurementhad the same reaction, and asked others to check their calculations and attempt toreplicate the result.What could go wrong in the measurement? You might think measuring speed is a simplematter of dividing distance by time. It is, at the scale and energy you live at. But witha fundamental particle like a neutrino, if you measure when it starts its journey, you stopthe journey. e particle is consumed by the measurement. So more subtle approaches areneeded. e detected difference from lightspeed, furthermore, is quite small, and so eventhe latency of the time it takes a reading to travel from a detector to a control room can beorders of magnitude larger. And since the “measurement” in this case is really an estimatefrom a statistical model, all of the assumptions of the model are now suspect. As I write thisin 2013, the physics community is unanimous that the FTL neutrino result was measurementerror. ey found the technical error, which involved a poorly attached cable, amongother things. 12 Furthermore, neutrinos clocked from supernova events are consistent withEinstein, and those distances are much larger and so would reveal differences in speed muchbetter.In both the woodpecker and neutrino dramas, the key dilemma is whether the falsificationis real or spurious. Measurement is complicated in both cases, but in quite differentways, rendering both true-detection and false-detection plausible. Popper himself was awareof this limitation inherent in measurement, and it may be one reason that Popper himselfsaw science as being broader than falsification. But the probabilistic nature of evidence rarelyappears when practicing scientists discuss the philosophy and practice of falsification. 13 Myreading of the history of science is that these sorts of measurement problems are the norm,not the exception. 141.2.2.2. Continuous hypotheses. Another problem for the swan story is that most interestingscientific hypotheses are not of the kind “all swans are white” but rather of the kind:Or maybe:H 0 : 80% of swans are white.H 0 : Black swans are rare.Now what are we to conclude, aer observing a black swan? e null hypothesis doesn’tsay black swans do not exist, but rather that they have some frequency. e task here isnot to disprove or prove a hypothesis of this kind, but rather to estimate and explain thedistribution of swan coloration as accurately as we can. Even when there is no measurementerror of any kind, this problem will prevent us from applying the modus tollens swan storyto our science. 15You might object that the hypothesis above is just not a good scientific hypothesis, becauseit isn’t easy to disprove. But if that’s the case, then most of the important questionsabout the world are not good scientific hypotheses. In that case, we should conclude that thedefinition of a “good hypothesis” isn’t doing us much good. Now, nearly everyone agreesthat it is a good practice to design experiments and observations that can differentiate competinghypotheses. But in many cases, the comparison must be probabilistic, a matter ofdegree, not kind. 161.2.3. Falsification is consensual. e scientific community does come to regard some hypothesesas false. e caloric theory of heat and the geocentric model of the universe are no