Research map / Learning theory and algorithms / Prediction and decision losses
A Online Statistical Framework for Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reframes OOD detection as online multiple hypothesis testing and proposes g-LOND, a generalized LOND procedure that controls FDR without assuming p-value independence and reports lower FPR than OOD baselines on CIFAR-100 and ImageNet-200.
Problem
Existing OOD detectors focus on score functions and use empirical threshold rules (TPR=95% on ID validation) that lack statistical guarantees; sequential decisions on streamed test inputs need error control when empirical p-values are dependent.
Approach
Treats each test input as a hypothesis with an empirical p-value from a calibration set and applies a generalized LOND online FDR procedure (g-LOND) with a function f in a class that guarantees FDR control regardless of p-value dependence, plus an asymptotic FPR→0 analysis under generalized Gaussian-like tails.
Key results
- On CIFAR-100 with Place365 OOD, g-LOND reduces FPR from 45.63% (best baseline) to 30.62% and raises F1 from 42.85% to 52.87%, at under 3% TPR cost.
- On ImageNet-200 with SSB-hard OOD, g-LOND reduces FPR from 67.06% to 53.72% and raises F1 from 37.87% to 49.31%, with 3.21% TPR drop.
- g-LOND reports lower FPR95 than baselines on ImageNet-200 (e.g., 9.4% better than best baseline on NINCO) and slightly higher AUROC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use g-LOND instead of the strongest baseline by FPR (KLM) | OOD detection with CIFAR-100 ID data · CIFAR-100 ID / CIFAR-10 OOD | FPR: 56.06 -> 44.19 | Helped |
| Use g-LOND instead of the strongest baseline by F1 (PALM) | OOD detection with CIFAR-100 ID data · CIFAR-100 ID / CIFAR-10 OOD | F1: 67.17 -> 68.08 | Helped |
| Use g-LOND instead of the strongest baseline by FPR (KLM) | OOD detection with CIFAR-100 ID data · CIFAR-100 ID / TinyImageNet OOD | FPR: 57.54 -> 53.77 | Helped |
| Use g-LOND instead of the strongest baseline by F1 (KNN) | OOD detection with CIFAR-100 ID data · CIFAR-100 ID / TinyImageNet OOD | F1: 76.74 -> 78.36 | Helped |
| Use g-LOND instead of the strongest baseline by FPR (Gram) | OOD detection with CIFAR-100 ID data · CIFAR-100 ID / SVHN OOD | FPR: 35.54 -> 29.76 | Helped |
| Use g-LOND instead of the strongest baseline by F1 (Gram) | OOD detection with CIFAR-100 ID data · CIFAR-100 ID / SVHN OOD | F1: 63.71 -> 69.94 | Helped |
| Use g-LOND instead of the strongest baseline by FPR (KLM) | OOD detection with ImageNet-200 ID data · ImageNet-200 ID / SSB-hard OOD | FPR: 67.06 -> 53.72 | Helped |
| Use g-LOND instead of the strongest baseline by F1 (KLM) | OOD detection with ImageNet-200 ID data · ImageNet-200 ID / SSB-hard OOD | F1: 37.87 -> 49.31 | Helped |
| Use g-LOND instead of the strongest baseline by FPR (KLM) | OOD detection with ImageNet-200 ID data · ImageNet-200 ID / Texture OOD | FPR: 39.86 -> 39.16 | Helped |
| Use g-LOND instead of the strongest baseline by F1 (KNN) | OOD detection with ImageNet-200 ID data · ImageNet-200 ID / Texture OOD | F1: 70.14 -> 74.86 | Helped |
Surprising findings
- Tables show g-LOND with lower TPR than most baselines on several benchmarks, so gains come from a trade-off rather than uniform improvement.
- Paper states its FPR-to-zero result holds for the asymptotic regime, while experiments are finite-sample on standard benchmarks.
Limitations
- FPR result is asymptotic under a generalized Gaussian-like assumption on test statistics, not verified in finite samples.
- Gains in TPR trade-off are not uniform; TPR drops on several OOD sets.
- Experiments rely on a single backbone family (ResNet) and the baselines follow the OpenOOD-style setup of prior work.
Experimental setup
Datasets: CIFAR-100 ImageNet-200
Benchmarks: CIFAR-100 with CIFAR-10, TinyImageNet, SVHN, Texture, Place365 as OOD ImageNet-200 with SSB-hard, NINCO, iNaturalist, OpenImage-O, Textures as OOD
Metrics: TPR FPR F1 FPR95 AUROC AUPR
Baselines: MSP ODIN Energy Gram VIM KNN KLM RankFeat DICE ASH Cider SHE
Builds on: LOND Benjamini-Hochberg procedure PRDS dependence alpha investing
Closest papers by meaning
- A Provable Decision Rule for Out-of-Distribution Detection (ICML 2024 · similarity 0.93)
- A Closer Look at Generalized BH Algorithm for Out-of-Distribution Detection (ICML 2025 · similarity 0.89)
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.87)
- Out-Of-Distribution Detection with Diversification (Provably) (NeurIPS 2024 · similarity 0.87)
- ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection (ICLR 2024 · similarity 0.87)
- Test-Time Linear Out-of-Distribution Detection (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.