Looking Elsewhere and Looking Everywhere (a.k.a. Multiple Testing ...

The Look Elsewhere Effect
Two “definitions” found in a (admittedly limited) search:
When searching for a new resonance somewhere in a possible mass range,
the significance of observing a local excess of events must take into
account the probability of observing such an excess anywhere in the range
OR
In experiments that are aimed at detecting astrophysical sources [...] one
usually performs a search over a continuous parameter space [...] looking
for the most significant deviation from the background hypothesis. Such a
procedure inherently involves a “look elsewhere effect”, namely, the
possibility for a signal-like fluctuation to appear anywhere within the
search range.
Victor M. Panaretos (EPFL) Progress on Statistical Issues in Searches SLAC – June 2012 2 / 25