Research map / Label-efficient and robust learning / OOD and anomaly detection
ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
Han Yu, Kehan Li, Dongbai Li, Yue He, Xingxuan Zhang, Peng Cui
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train models with RSC instead of ERM (ablation) | performance prediction algorithms · PACS | average Spearman rho: 0.553 -> 0.595 | Improved |
| train models with Mixup instead of ERM (ablation) | performance prediction algorithms · PACS | average Spearman rho: 0.553 -> 0.620 | Improved |
| train models with SWAD instead of ERM (ablation) | performance prediction algorithms · PACS | average Spearman rho: 0.553 -> 0.569 | Improved |
| train models with CORAL instead of ERM (ablation) | performance prediction algorithms · PACS | average Spearman rho: 0.553 -> 0.569 | Improved |
| train models with RSC instead of ERM (ablation) | MaNo · PACS | Spearman rho: 0.489 -> 0.788 | Improved |
| train models with Mixup instead of ERM (ablation) | COTT · PACS | Spearman rho: 0.636 -> 0.831 | Improved |
| use MoCo/CLIP instead of supervised pretraining (ablation) | ATC, NI, COTT · PACS | Spearman rho | Improved |
| use supervised instead of MoCo/CLIP pretraining (ablation) | Agreement · PACS | Spearman rho | Improved |
| switch from ResNet-50 to larger ViT-B/16 (ablation) | performance prediction algorithms (except ATC) · PACS | Spearman rho | Hurt |
| increase corruption severity (ablation) | NI · ImageNet-C and TinyImageNet-C | Spearman rho | Hurt |
| evaluate with rho@10% instead of rho (ablation) | MaNo · ImageNet-A | ranking metric: 0.417 -> 0.927 | Improved |
| evaluate on natural shift (CelebA) instead of synthetic corruption (CIFAR-10-C) | average of 10 algorithms · OOD datasets | Spearman rho: 0.746 -> 0.042 | Hurt |
Closest papers by meaning
- Enhancing the Power of OOD Detection via Sample-Aware Model Selection (CVPR 2024 · similarity 0.86)
- Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations (NeurIPS 2025 · similarity 0.86)
- OODD: Test-time Out-of-Distribution Detection with Dynamic Dictionary (CVPR 2025 · similarity 0.85)
- ITP: Instance-Aware Test Pruning for Out-of-Distribution Detection (AAAI 2025 · similarity 0.85)
- Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution (NeurIPS 2024 · similarity 0.85)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
