Research map / Probabilistic and scientific modeling / Molecules and proteins

Finite Resources False Discovery Rate Control on Structured Hypothesis Spaces

NeurIPS 2026 · Probabilistic and scientific modeling › Molecules and proteins · approach: Statistical analysis of ML practice

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes count-space, RKHS-regularized FDR control for hypotheses tested with finitely many null draws, with two decision rules (gated Sun–Cai and count-space mirror statistics) plus a sample-allocation policy.

Problem

Large-scale hypothesis testing with only finitely many null (reference) samples per hypothesis, leaving p-values uncertain, while hypotheses may have spatial/structured dependence; FDR must still be controlled.

Approach

Marginalizes the latent p-value so each hypothesis contributes a Beta-Binomial count likelihood; learns a null-probability surface alpha over an RKHS kernel geometry; uses a count-space mirror (k ↔ m−k) with Barber–Candès thresholding, accepting an approximate-mirror slack, and a variance-floor-aware allocation policy for null draws.

Key results

• On the AlpacaEval 2.0 setup (N=805, m=6), Rule 2 makes 168 discoveries at FDR 0.042 and power 0.315, versus 48 discoveries for BH on the heuristic p-values (FDR 0.021, power 0.092).
• Across 400 configurations on 10 ADBench datasets, both proposed rules keep FDR below the nominal 0.1 target, while BH and RegFDR with point-estimated p-values violate FDR control in the finite-m regime.
• Active allocation makes more decisions per unit budget and reaches higher power than uniform and random allocation on ADBench, with target tau=0.15 achieved at saturation but exceeded transiently for small budgets.

Surprising findings

• Plug-in RegFDR, used as the oracle labeler, itself violates FDR control when fed point-estimated p-values at finite m.
• The BH baseline on standard discrete p-values cannot make any rejection at m=6 because the minimum p-value 1/(m+1) exceeds the FDR target.

Limitations

• Rule 2's exact FDR guarantee relies on a flip-stability assumption; with the deployed plug-in fit it holds only up to a slack.
• Assumes a uniform theoretical null; empirical nulls are only handled by a proposed Beta-mixture extension not evaluated.
• Active allocation violates the i.i.d. assumption at small sample sizes, requiring a burn-in phase; the FDR target is exceeded transiently for small budgets.

Experimental setup

Datasets: ADBench AlpacaEval 2.0

Benchmarks: ADBench AlpacaEval 2.0

Metrics: FDR power number of discoveries RKHS distance to oracle alpha

Baselines: Benjamini-Hochberg RegFDR with point-estimated p-values uncalibrated heuristic BH on 1 - wins/m Uniform allocation Random allocation

Builds on: RegFDR Barber-Candès knockoff filter Sun-Cai lfdr threshold SmoothFDR AdaPT

Closest papers by meaning

• Calibration without labels in multiple testing (NeurIPS 2026 · similarity 0.86)
• Foundations of Testing for Finite-Sample Causal Discovery (ICML 2024 · similarity 0.84)
• MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search (NeurIPS 2025 · similarity 0.84)
• Optimal Experiments for Partial Causal Effect Identification (NeurIPS 2026 · similarity 0.83)
• Conformalized Multiple Testing after Data-dependent Selection (NeurIPS 2024 · similarity 0.83)
• Strong and Weak Identifiability of Optimization-based Causal Discovery in Non-linear Additive Noise Models (ICML 2025 · similarity 0.83)