Looking Elsewhere and Looking Everywhere (a.k.a. Multiple Testing ...

Taking Advantage of Structure: Dependence
Though independence provides some improvement over the Holm and
Bonferroni methods, in some sense it complicates things rather than
simplifies them.
Dependence can complicate the analysis, but improve power considerably!
Manifests itself in any of the two following ways (or a combination)
1 Dependence between the test statistics for each H0,t
↩→ Can arise if the measurement errors are correlated, or if the datasets on
which individual test statistics Wt are based on are not mutually
exclusive for different values of t = 1, ..., T .
2 Logical dependence between the hypotheses H0,t
↩→ It might be that rejecting a hypothesis H0,t1 would imply necessarily
the rejection of another, H0,t2 , or, more generally, that not all
combinations of rejections of individual hypotheses are feasible
(effective number of individual tests smaller than T ).
Victor M. Panaretos (EPFL) Progress on Statistical Issues in Searches SLAC – June 2012 13 / 25