Research map / Label-efficient and robust learning / OOD and anomaly detection
Out-of-Distribution Detection via Deep Multi-Comprehension Ensemble
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Multi-Comprehension (MC) Ensemble, which ensembles ResNet models trained with different supervision tasks (CE, SimCLR, SupCon) rather than different initializations, and uses a Self-Coupling Index and loss-barrier analysis to show this yields more diverse features and better OOD detection.
Problem
Deep ensembles trained with the same criterion (different initializations or data subsets) provide limited feature-representation diversity, so they give little OOD detection gain over a single model of equal size.
Approach
Diversity comes from the training task: ensemble members trained with different supervision losses (cross-entropy, SimCLR, SupCon) fall into distant loss basins and produce less self-coupled penultimate features, so averaging their features boosts OOD detection. A Self-Coupling Index based on Sinkhorn optimal transport quantifies this diversity.
Key results
- On CIFAR10, MC Ensemble with KNN scoring reaches 5.58% average FPR95 and 98.73% AUROC, versus 30.77% / 94.15% for a single SupCE model.
- MC Ensemble (5.58 FPR95) outperforms the naive 3xSupCE ensemble (27.78 FPR95) and matches or beats ResNet-62 (23.79 FPR95) with the same parameter count.
- On ImageNet, MC Ensemble averages 30.69% FPR95 and 93.06% AUROC, versus 35.67% / 91.64% for the best naive ensemble (3xSupCon).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine models trained with SupCE, SimCLR, and SupCon | ResNet-18 OOD detector with KNN scoring · CIFAR10 OOD average | FPR95: 11.07 -> 5.58 | Helped |
| combine models trained with SupCE, SimCLR, and SupCon | ResNet-18 OOD detector with KNN scoring · CIFAR10 OOD average | AUROC: 97.93 -> 98.73 | Helped |
| combine models trained with SupCE, SimCLR, and SupCon | ResNet-18 OOD detector · CIFAR10 vs CIFAR100 | FPR95: 36.72 -> 23.35 | Helped |
| combine models trained with SupCE, SimCLR, and SupCon | ResNet-50 OOD detector with KNN scoring · ImageNet-1K OOD average | FPR95: 35.67 -> 30.69 | Helped |
| use two SupCon models instead of three distinct training tasks (ablation) | MC Ens. · CIFAR10 OOD average | FPR95: 5.58 -> 12.34 | Hurt |
| omit SupCon and combine SupCE with SimCLR (ablation) | MC Ens. · CIFAR10 OOD average | FPR95: 5.58 -> 24.98 | Hurt |
| omit SimCLR and combine SupCE with SupCon (ablation) | MC Ens. · CIFAR10 OOD average | FPR95: 5.58 -> 9.37 | Hurt |
| omit SupCE and combine SimCLR with SupCon (ablation) | MC Ens. · CIFAR10 OOD average | FPR95: 5.58 -> 9.42 | Hurt |
| replace MC Ens. with a single comparable-scale ResNet-62 (ablation) | CIFAR10 OOD detector · CIFAR10 OOD average | FPR95: 5.58 -> 23.79 | Hurt |
| replace MC Ens. with a same-task 3×SupCE ensemble (ablation) | CIFAR10 OOD detector · CIFAR10 OOD average | FPR95: 5.58 -> 27.78 | Hurt |
| distill MC Ens. into one model (ablation) | MC Ens. · CIFAR10 OOD average | FPR95: 5.58 -> 8.17 | Hurt |
Surprising findings
- Naive ensembles of same-criterion models give little or no OOD gain over single models; a 3xSimCLR naive ensemble even underperforms its single SimCLR model in AUROC on CIFAR10.
- Models trained on 80% data subsets with different initializations remain highly self-coupled, indicating data partitioning adds little feature diversity.
Limitations
- Ensemble incurs M-times inference compute and memory compared to a single model.
- Ensemble member selection via the proposed SCI objective is only formulated; the instantiated ensemble uses three hand-picked losses.
- Evaluated only on CIFAR10 and ImageNet with ResNet backbones; ViT results only in appendix.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet-1K SVHN iSUN LSUN Texture Places365 SUN iNaturalist
Benchmarks: CIFAR10 OOD benchmark (SVHN, LSUN, iSUN, Texture, Places365) CIFAR10 vs CIFAR100 ImageNet OOD benchmark (iNaturalist, SUN, Places, Textures)
Metrics: AUROC FPR95
Baselines: Deep Ensemble (3xSupCE) ODIN MSP Mahalanobis Energy KNN CSI SSD+ KNN+ FeatureNorm DOE CIDER
Builds on: Deep Ensembles Git Re-Basin SupCon SimCLR KNN OOD detection EnD^2
Closest papers by meaning
- Spurious Feature Diversification Improves Out-of-distribution Generalization (ICLR 2024 · similarity 0.87)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.86)
- Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance (NeurIPS 2024 · similarity 0.86)
- Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting (ICML 2024 · similarity 0.86)
- Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection (ICLR 2025 · similarity 0.86)
- Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.