Research map / Label-efficient and robust learning / OOD and anomaly detection
Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Split-Ensemble, which splits a multiclass task into complementary feature-grouped subtasks with OOD-aware class-balanced BCE training and builds a tree-like shared-backbone ensemble via iterative IoU-based splitting and Hessian-based pruning, improving accuracy and OOD detection at single-model FLOPs without external OOD data.
Problem
Deep classifiers give poorly calibrated uncertainty on OOD inputs; good OOD detection usually needs external OOD data for outlier exposure or costly deep ensembles that multiply inference compute.
Approach
Split the N-class task into complementary subtasks grouped by feature similarity, where each submodel treats other classes as an OOD class (trained with outlier-exposure-style soft targets and class-balanced BCE), and share early layers across submodels with a tree-like architecture found by iterative correlation-based splitting (IoU of SNIP sensitivity masks, minimal cutting threshold) and Hessia
Key results
- Split-Ensemble improves accuracy over a single ResNet-18 by 0.8% on CIFAR-10, 1.8% on CIFAR-100, and 25.5% on Tiny-ImageNet at the same FLOPs.
- Mean OOD AUROC for CIFAR-10 ID improves to 93.7 vs 91.5 for single model and 91.1 for 4x naive ensemble.
- On ImageNet1K, Split-Ensemble reaches 70.9% top-1 vs 69.0% single model and 69.4% naive ensemble.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OOD-aware rather than one-hot OOD-class targets (ablation) | Split-Ensemble · CIFAR-100 ID; CIFAR-10 OOD; 2, 4, and 5 splits | AUROC: 76.0 -> 78.1; 77.5 -> 78.2; 77.3 -> 78.1 | Helped |
| use OOD-aware rather than one-hot OOD-class targets (ablation) | Split-Ensemble · CIFAR-100 ID; 2, 4, and 5 splits | accuracy: 71.0 -> 77.7; 77.2 -> 78.0; 77.7 -> 77.9 | Helped |
| raise the MCT threshold from 0.0 (all-share) to 0.2 (ablation) | Split-Ensemble with 5 subtask splits · CIFAR-100 | accuracy: 76.2 -> 78.4 | Helped |
| raise the MCT threshold from 0.0 (all-share) to 0.4 (ablation) | Split-Ensemble with 5 subtask splits · CIFAR-100 ID; CIFAR-10 OOD | AUROC: 76.7 -> 79.9 | Helped |
| increase subtask splits from 2 to 8 (ablation) | Split-Ensemble · CIFAR-100 ID; CIFAR-10 OOD | AUROC: 78.1 -> 80.4 | Helped |
| increase subtask splits from 8 to 10 (ablation) | Split-Ensemble · CIFAR-100 ID; CIFAR-10 OOD | AUROC: 80.4 -> 77.3 | Hurt |
| group classes semantically rather than randomly (ablation) | Split-Ensemble · CIFAR-100 ID; CIFAR-10 OOD; 2 splits | AUROC: 78.9 -> 79.6 | Helped |
| group classes semantically rather than randomly (ablation) | Split-Ensemble · CIFAR-100 ID; CIFAR-10 OOD; 5 splits | AUROC: 77.3 -> 78.9 | Helped |
| use Split-Ensemble rather than a single model | ResNet-18 classifier · CIFAR-10 | accuracy: 94.7 -> 95.5 | Helped |
| use Split-Ensemble rather than a single model | ResNet-18 classifier · CIFAR-100 | accuracy: 75.9 -> 77.7 | Helped |
| use Split-Ensemble rather than a single model | ResNet-18 classifier · TinyImageNet | top-1 accuracy: 26.1 -> 51.6 | Helped |
| use Split-Ensemble rather than a single model | ResNet-18 classifier · ImageNet1K | top-1 accuracy: 69.0 -> 70.9 | Helped |
Surprising findings
- On CIFAR-100 OOD detection against CIFAR-10, the single model and naive ensemble achieve higher AUROC on some OOD sets; Split-Ensemble is not uniformly best (e.g., FPR on TinyImageNet crop for CIFAR-10 ID is worse than single model).
Limitations
- Evaluated only on small-scale image classification benchmarks (CIFAR, Tiny ImageNet, ImageNet1K); not tested on large models such as LLMs.
- Number of subtask splits and class grouping are chosen heuristically via ablation rather than learned automatically.
- Aggressive pruning with many branches or large MCT threshold hurts performance.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet1K CIFAR10-LT CIFAR100-LT SVHN LSUN Gaussian noise Uniform noise CIFAR-10-C TinyImageNet (crop/resize)
Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet1K SC-OOD CIFAR10-LT CIFAR-10-C
Metrics: accuracy AUROC AUPR FPR at 95% TPR detection error ECE
Baselines: Single Model Naive Ensemble MC-Dropout MIMO MaskEnsemble BatchEnsemble FilmEnsemble ODIN EBO OE MCD UDG
Builds on: outlier exposure SNIP Hessian-based structural pruning class-balanced loss deep ensembles
Closest papers by meaning
- Out-of-Distribution Detection via Deep Multi-Comprehension Ensemble (ICML 2024 · similarity 0.86)
- Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution (ICML 2025 · similarity 0.85)
- Ensemble Pruning for Out-of-distribution Generalization (ICML 2024 · similarity 0.85)
- Spurious Feature Diversification Improves Out-of-distribution Generalization (ICLR 2024 · similarity 0.85)
- Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance (NeurIPS 2024 · similarity 0.84)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.