The Benjamini–Hochberg Procedure Can Fail to Control the FDR for Correlated Two-Sided Gaussian Tests

For decades, it has been widely conjectured that the Benjamini-Hochberg (BH) procedure, the standard method for controlling the false discovery rate (FDR) in multiple hypothesis testing, maintains its nominal level for two-sided Gaussian tests regardless of the correlation structure. This paper investigates whether this conjecture holds or if the BH procedure can fail to control the FDR in the presence of specific correlations.

The author constructs a Gaussian factor model where the null hypotheses and signal hypotheses are correlated through a latent factor. By conditioning on this latent factor, the author derives a limiting empirical distribution of p-values. The proof utilizes a rigorous interval-arithmetic certificate (using Arb) to verify that the BH threshold leads to an FDR exceeding the nominal level of 0.01. The argument relies on bracketing the BH threshold between two strictly defined values, avoiding the need for complex convergence proofs of the threshold itself.

The study provides a formal proof that the BH procedure fails to control the FDR at the nominal level for correlated two-sided Gaussian tests. Specifically, for a constructed factor model at alpha = 0.01, the FDR is proven to be strictly greater than 0.0104 for sufficiently large numbers of hypotheses. This theoretical result is supported by stratified Monte Carlo simulations, which show the empirical FDR exceeding the nominal level as the number of hypotheses increases.

This finding resolves a twenty-year-old open problem in statistics, overturning a widely held belief that the BH procedure was robust to arbitrary correlation structures in two-sided Gaussian settings. It highlights the potential risks of applying standard FDR control methods in complex, correlated data environments—such as genomics or finance—and suggests that more conservative or dependence-adjusted procedures may be necessary in certain high-dimensional applications.