Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
URL: A Representation Learning Benchmark for Transferable Uncertainty Estimates
Michael Kirchhof, Bálint Mucsányi, Seong Joon Oh, Enkelejda Kasneci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Zero-shot transferred uncertainty versus many-shot training on downstream classes | All 11 uncertainty quantifiers · URL benchmark | R-AUROC: 0.68 | Hurt |
| Zero-shot pretrained versus many-shot downstream training | ResNet-50 R@1 · Downstream datasets | R@1: 0.48 vs. 0.54 | Hurt |
| Choose hyperparameters for R-AUROC instead of R@1 (ablation) | HIB on ViT · URL benchmark | R@1 vs R-AUROC: -0.21 R@1 for +0.01 R-AUROC | Mixed |
| Hyperparameter sweep: correlation of R@1 and R-AUROC across settings (ablation) | 22 model-backbone combinations · URL benchmark | R-AUROC vs R@1: 15 out of 22 | Mixed |
| Use upstream R-AUROC to predict downstream R-AUROC | All models and hyperparameters · ImageNet to CUB/CARS/SOP | rank correlation: 0.09 | No clear effect |
| Use cropped-image uncertainty fraction as a model selection proxy | Pretrained uncertainty models · Downstream R-AUROC | rank correlation: 0.54 | Improved |
| R-AUROC as proxy for human-uncertainty alignment | All models · Five human-annotated datasets | rank correlation: 0.80 | Improved |
| R-AUROC versus cropping detection | All models · Cropped images | rank correlation: 0.71 | Improved |
| Use class entropy or embedding norm as uncertainty estimate for OOD detection | InfoNCE, CE, SNGP, HET-XL, Ensemble · ImageNet vs CUB/CARS/SOP | OOD AUROC: 0.73 | Improved |
| Ensembles and MCDropout as uncertainty quantifiers | Deep Ensemble, MCDropout · URL benchmark | R-AUROC | Hurt |
| Direct variance modeling (Losspred, SNGP) versus other methods | ViT backbones · URL benchmark | R-AUROC | Improved |
| Mixed ID+OOD data instead of OOD only | HET-XL and SNGP · ImageNet+CUB/CARS/SOP mixtures | R-AUROC | Improved |
Closest papers by meaning
- Transitional Uncertainty with Layered Intermediate Predictions (ICML 2024 · similarity 0.86)
- probly: Uncertainty-Aware Machine Learning (NeurIPS 2026 · similarity 0.86)
- Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks (NeurIPS 2024 · similarity 0.85)
- REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability (AAAI 2025 · similarity 0.85)
- Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere (NeurIPS 2025 · similarity 0.85)
- Uncertainty Estimation by Flexible Evidential Deep Learning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
