AI Paper Atlas

Research map / Label-efficient and robust learning / Test-time adaptation

Predicting With Confidence on Unseen Distributions

Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, Ludwig Schmidt

ICCV 2021 · Label-efficient and robust learning › Test-time adaptation · approach: Robust optimization for distribution shift · 104 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
predict accuracy with DoC instead of the average-confidence baseline ACImageNet classifiers · Natural shifts (calibrated on Syn1)MAE: 0.087 -> 0.050Improved
use DoC instead of naive base accuracyImageNet classifiers · Natural shifts (calibrated on Syn1)MAE: 0.234 -> 0.050Improved
keep only maximum softmax probability (DoC) instead of entropy (DoE) (ablation)DoC · Natural shifts (calibrated on Syn1)MAE: 0.075 -> 0.050Improved
use Frechet distance instead of ACImageNet classifiers · Natural shifts (calibrated on Syn1)MAE: 0.087 -> 0.113Hurt
use discriminative A-proxy distance instead of ACImageNet classifiers · Natural shifts (calibrated on Syn1)MAE: 0.087 -> 0.335Hurt
use DoC instead of AC on synthetic-to-syntheticImageNet classifiers · Synthetic Syn2 shifts (calibrated on Syn1)MAE: 0.075 -> 0.039Improved
use DoC instead of Frechet distance on synthetic-to-syntheticImageNet classifiers · Synthetic Syn2 shifts (calibrated on Syn1)MAE: 0.0384 -> 0.039No clear effect
use DoC instead of AC on adversarial ImageNet-AImageNet classifiers · ImageNet-A (calibrated on Syn1)MAE: 0.476 -> 0.388Improved
calibrate DoC on natural shifts instead of Syn1 for ImageNet-A (ablation)DoC · ImageNet-AMAE: 0.388 -> 0.292Improved
calibrate DoE on natural shifts instead of Syn1 for ImageNet-A (ablation)DoE · ImageNet-AMAE: 0.524 -> 0.371Improved
calibrate DoC on Syn2 instead of Syn1 (ablation)DoC · Natural shiftsMAE: 0.050 -> 0.066Hurt
calibrate DoC on natural shifts for synthetic target (ablation)DoC · Synthetic Syn2 shiftsMAE: 0.039 -> 0.096Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy