AI Paper Atlas

Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

URL: A Representation Learning Benchmark for Transferable Uncertainty Estimates

Michael Kirchhof, Bálint Mucsányi, Seong Joon Oh, Enkelejda Kasneci

NeurIPS 2023 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Zero-shot robustness and transfer · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Zero-shot transferred uncertainty versus many-shot training on downstream classesAll 11 uncertainty quantifiers · URL benchmarkR-AUROC: 0.68Hurt
Zero-shot pretrained versus many-shot downstream trainingResNet-50 R@1 · Downstream datasetsR@1: 0.48 vs. 0.54Hurt
Choose hyperparameters for R-AUROC instead of R@1 (ablation)HIB on ViT · URL benchmarkR@1 vs R-AUROC: -0.21 R@1 for +0.01 R-AUROCMixed
Hyperparameter sweep: correlation of R@1 and R-AUROC across settings (ablation)22 model-backbone combinations · URL benchmarkR-AUROC vs R@1: 15 out of 22Mixed
Use upstream R-AUROC to predict downstream R-AUROCAll models and hyperparameters · ImageNet to CUB/CARS/SOPrank correlation: 0.09No clear effect
Use cropped-image uncertainty fraction as a model selection proxyPretrained uncertainty models · Downstream R-AUROCrank correlation: 0.54Improved
R-AUROC as proxy for human-uncertainty alignmentAll models · Five human-annotated datasetsrank correlation: 0.80Improved
R-AUROC versus cropping detectionAll models · Cropped imagesrank correlation: 0.71Improved
Use class entropy or embedding norm as uncertainty estimate for OOD detectionInfoNCE, CE, SNGP, HET-XL, Ensemble · ImageNet vs CUB/CARS/SOPOOD AUROC: 0.73Improved
Ensembles and MCDropout as uncertainty quantifiersDeep Ensemble, MCDropout · URL benchmarkR-AUROCHurt
Direct variance modeling (Losspred, SNGP) versus other methodsViT backbones · URL benchmarkR-AUROCImproved
Mixed ID+OOD data instead of OOD onlyHET-XL and SNGP · ImageNet+CUB/CARS/SOP mixturesR-AUROCImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy