Research map / Label-efficient and robust learning / OOD and anomaly detection
A Provable Decision Rule for Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes g-BH, a generalized Benjamini-Hochberg multiple-testing decision rule with empirical p-values that replaces the threshold-based OOD decision rule, with FDR control and an FPR bound, and plugs into existing score functions.
Problem
OOD detection decision rules are empirical threshold rules (TPR=95% on ID validation) with no rigorous guarantee; the paper asks how to design a decision rule with theoretical guarantees and good empirical performance.
Approach
Frame OOD detection as multiple hypothesis testing over test samples and apply a generalized BH procedure (f in F1 or F2 applied to empirical p-values from a calibration set of ID data) that controls FDR; additionally bound expected FPR under a tailed generalized Gaussian model.
Key results
- On CIFAR-10 vs Places365 with KNN score, g-BH reduces FPR from 48.21% to 16.79% and raises F1 from 49.83% to 67.86% versus the empirical decision rule.
- Plugging MSP into g-BH reduces FPR95 by 13.65% on average compared with vanilla MSP across OOD sets.
- g-BH is shown to control FDR at level n0/n * alpha under independence or PRDS, and the expected FPR bound tends to zero as n grows.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply g-BH decision rule to KNN | KNN with ResNet-18 · CIFAR-100 | FPR: 55.17 -> 28.32 | Helped |
| Apply g-BH decision rule to KNN | KNN with ResNet-18 · TinyImageNet | FPR: 50.59 -> 21.11 | Helped |
| Apply g-BH decision rule to KNN | KNN with ResNet-18 · SVHN | FPR: 52.75 -> 14.26 | Helped |
| Apply g-BH decision rule to KNN | KNN with ResNet-18 · Texture | FPR: 45.98 -> 19.55 | Helped |
| Apply g-BH decision rule to KNN | KNN with ResNet-18 · Place365 | FPR: 48.21 -> 16.79 | Helped |
| Apply g-BH decision rule to MSP | MSP with ResNet-18 · SVHN | FPR95: 53.22 -> 38.75 | Helped |
Surprising findings
- The g-BH decision rule improves FPR and F1 across all eight score functions even though it trades a slight drop in TPR.
Limitations
- FDR guarantee relies on independence or PRDS of p-values, which is assumed for neural score-based p-values rather than proven in general.
- The FPR bound relies on a tailed generalized Gaussian model and sparse-regime assumptions that may not match real score distributions.
- Experiments are limited to CIFAR-10 as the ID dataset with a single ResNet-18 backbone.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet SVHN Texture Places365 MNIST
Benchmarks: CIFAR-10 vs CIFAR-100 CIFAR-10 vs TinyImageNet CIFAR-10 vs SVHN CIFAR-10 vs Texture CIFAR-10 vs Places365 CIFAR-10 vs MNIST
Metrics: TPR FPR F1-score FPR95 AUROC AUPR
Baselines: MSP KLM KNN LogitNorm RankFeat ASH Cider GEN
Builds on: Benjamini-Hochberg procedure PRDS dependence condition (Benjamini & Yekutieli) OpenOOD benchmark
Closest papers by meaning
- A Online Statistical Framework for Out-of-Distribution Detection (ICML 2025 · similarity 0.93)
- A Closer Look at Generalized BH Algorithm for Out-of-Distribution Detection (ICML 2025 · similarity 0.91)
- Test-Time Linear Out-of-Distribution Detection (CVPR 2024 · similarity 0.86)
- ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection (ICLR 2024 · similarity 0.86)
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.86)
- What If the Input is Expanded in OOD Detection? (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.