Research map / Probabilistic and scientific modeling / Variational and generative sampling
ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConjNorm, a Bregman-divergence-based density framework for post-hoc OOD detection that searches for an optimal l_p norm coefficient and estimates the partition function via importance sampling.
Problem
Post-hoc out-of-distribution detection needs a density score that accurately tracks the true ID data density, but normalization constants are costly or intractable and prior methods impose strong distributional assumptions (Gaussian, Gibbs-Boltzmann).
Approach
Connects exponential family distributions with Bregman divergence via conjugate Legendre functions, so the density design reduces to choosing an l_p norm (with conjugate l_q) cumulant; estimates the partition function unbiasedly with importance sampling over ID training features.
Key results
- On CIFAR-100 (DenseNet), ConjNorm reaches 28.27% FPR95 and 92.50% AUROC, versus 41.40% FPR95 for the best prior baseline ASH.
- On ImageNet-1K with MobileNetV2, ConjNorm averages 21.51% FPR95 and 95.48% AUROC across four OOD datasets; adding ASH gives further gains.
- Hard OOD and long-tailed settings also show improvements; on the CIFAR-100 long-tailed setup ConjNorm gets 40.16% FPR95 on one of the OOD sets vs 64.39% for KNN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ConjNorm instead of ASH | DenseNet-101 OOD detection · CIFAR-10 (6 OOD datasets) | FPR95: 15.05 -> 13.92 | Helped |
| Use ConjNorm instead of ASH | DenseNet-101 OOD detection · CIFAR-10 (6 OOD datasets) | AUROC: 96.61 -> 97.15 | Helped |
| Use ConjNorm instead of ASH | DenseNet-101 OOD detection · CIFAR-100 (6 OOD datasets) | FPR95: 41.40 -> 28.27 | Helped |
| Use ConjNorm instead of ASH | DenseNet-101 OOD detection · CIFAR-100 (6 OOD datasets) | AUROC: 90.02 -> 92.50 | Helped |
| Add ASH to ConjNorm | ConjNorm · CIFAR-10 (6 OOD datasets) | FPR95: 13.92 -> 12.14 | Helped |
| Add ASH to ConjNorm | ConjNorm · CIFAR-100 (6 OOD datasets) | FPR95: 28.27 -> 25.66 | Helped |
| Use features from a deeper layer rather than shallower layers (ablation) | ConjNorm · CIFAR-10 | OOD detection performance | Helped |
| Vary the importance-sampling ratio, particularly using ratios at or above 10% (ablation) | ConjNorm · CIFAR-100 and ImageNet-1K | FPR95 | No clear effect |
| Reduce the importance-sampling ratio to 1% (ablation) | ConjNorm · CIFAR-100 and ImageNet-1K | OOD detection performance | No clear effect |
| Set the norm coefficient p to 2 rather than selecting the reported optimum in (2, 3) (ablation) | ConjNorm · CIFAR-10 and CIFAR-100 | FPR95 | Hurt |
| Use the conjugate q=p/(p-1) rather than q=2.0 (ablation) | ConjNorm · CIFAR-100 | FPR95: nearly 10% | Helped |
Surprising findings
- The Gaussian case p=2 (GEM-like) does not give the best OOD performance; the optimal norm coefficient lies around p in (2,3) on CIFAR.
- Removing activations via ASH on top of ConjNorm improves ImageNet results by about 9.27% FPR95, suggesting complementarity.
Limitations
- The norm coefficient p must be searched manually per dataset.
- Only tested on convolutional networks and small-to-medium models, not large-scale models.
- Only the conjugate l_p/l_q pair is considered among the space of Bregman divergences.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-1K SVHN LSUN-Crop LSUN-Resize iSUN Places365 Textures iNaturalist SUN
Metrics: FPR95 AUROC
Baselines: MSP ODIN Energy ASH DICE ReAct Mahalanobis GEM KNN SHE
Builds on: GEM Mahalanobis distance DICE energy-based OOD
Closest papers by meaning
- Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the Role of Model Complexity (NeurIPS 2025 · similarity 0.89)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.89)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.88)
- Out-of-Distribution Detection with Prototypical Outlier Proxy (AAAI 2025 · similarity 0.88)
- DOS: Diverse Outlier Sampling for Out-of-Distribution Detection (ICLR 2024 · similarity 0.88)
- A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.