Research map / Label-efficient and robust learning / OOD and anomaly detection
Provable Discriminative Hyperspherical Embedding for Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DHE, a prototype-based contrastive framework that pre-computes maximally dispersed hyperspherical class prototypes (via an angular spread loss) and then trains an encoder with a prototype-enhanced contrastive loss for distance-based OOD detection.
Problem
Distance-based OOD detection relies on discriminative embeddings, but existing objectives (SupCon, CIDER's iterative dispersion loss) are designed for classification and may get stuck in local optima of inter-class prototype dispersion, reducing separation of ID and OOD examples.
Approach
Theoretically show FPR on OOD data scales with intra-class distance over inter-class prototype distance; derive that the sum of squared inter-prototype distances is maximized when prototypes form a simplex (inner products 1/(1-K)); pre-compute such prototypes with an angular spread loss before training, then align ID embeddings to fixed prototypes with a PEC loss.
Key results
- On CIFAR-100 (ResNet-34, KNN score), DHE reaches 43.52% average FPR95 versus 48.89% for CIDER, a 5.37-point reduction.
- DHE's average AUROC on CIFAR-100 is 87.82%, above CIDER's 87.39%.
- Training time per epoch on CIFAR-100 is about 49.98 s for DHE versus 131.32 s for CIDER, a 62% reduction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DHE instead of CIDER with KNN scoring | CIFAR-100 OOD detection · Five OOD benchmarks average | FPR95: 5.37% | Helped |
| Use DHE instead of CIDER with KNN scoring | CIFAR-100 OOD detection · Five OOD benchmarks average | AUROC: 87.39 -> 87.82 | Helped |
| Use DHE instead of CIDER with Mahalanobis scoring | CIFAR-100 OOD detection · Five OOD benchmarks average | FPR95: 4.76% | Helped |
| Use DHE instead of CIDER with Mahalanobis scoring | CIFAR-100 OOD detection · Five OOD benchmarks average | AUROC: 87.98 -> 89.01 | Helped |
| Use DHE instead of CIDER | CIFAR-100 training with ResNet-34 · CIFAR-100 ID | training time (s/epoch): 62% | Helped |
| Use DHE instead of CE | CIFAR-100 training with ResNet-34 · CIFAR-100 ID | training time (s/epoch): 41.16 -> 49.98 | Hurt |
Limitations
- Theoretical guarantee covers prototype dispersion, while the FPR link relies on a distance-based scoring assumption.
- Large-scale experiments only fine-tune the last block of a pre-trained ResNet-50 on ImageNet-100, a limited setting.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-100
Benchmarks: SVHN Places365 Texture LSUN iSUN iNaturalist SUN
Metrics: FPR95 AUROC
Baselines: MSP ODIN Mahalanobis Energy GODIN CE+SimCLR CSI SSD+ KNN+ CIDER T2FNORM ReweightOOD
Builds on: SupCon CIDER KNN+ SSD+
Closest papers by meaning
- Learning to Shape In-distribution Feature Space for Out-of-distribution Detection (NeurIPS 2024 · similarity 0.90)
- Learning with Mixture of Prototypes for Out-of-Distribution Detection (ICLR 2024 · similarity 0.89)
- CASE: Exploiting Intra-class Compactness and Inter-class Separability of Feature Embeddings for Out-of-Distribution Detection (AAAI 2024 · similarity 0.89)
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.88)
- HYPO: Hyperspherical Out-Of-Distribution Generalization (ICLR 2024 · similarity 0.88)
- EAT: Towards Long-Tailed Out-of-Distribution Detection (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.