AI Paper Atlas

Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

Benchmarking Low-Shot Robustness to Natural Distribution Shifts

Aaditya Singh, Kartik Sarangmath, Prithvijit Chattopadhyay, Judy Hoffman

ICCV 2023 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Robust fine-tuning of zero-shot models · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use self-supervised ViTs instead of SSL CNNs and supervised models (ablation)ImageNet pre-trained low-shot models · ImageNet, iWildCam, CamelyonID/OOD accuracyImproved
choose MSN ViTB-16 vs DINO / ViTS-16 initialization (ablation)low-shot fine-tuning · ImageNet, iWildCam, CamelyonOOD accuracy: ImageNet OOD 22.81 (MSN ViTB) vs iWildCam 19.99 (MSN ViTS) vs Camelyon 85.09 (DINO ViTS)Mixed
use CLIP instead of ImageNet-1k pre-trained DINO ViTB-16 (ablation)low-shot ViTB-16 · iWildCam OODavg per-class accuracy: 19.82 -> 19.10Hurt
use CLIP instead of ImageNet-1k pre-trained DINO ViTB-16 (ablation)low-shot ViTB-16 · Camelyon OODavg per-class accuracy: 84.33 -> 77.3Hurt
use CLIP zero-shot instead of low-shot trained models (ablation)ImageNet · ImageNet OODtop-1 accuracy: 27.50 -> 57.37Improved
use IN21k supervised ViT instead of IN1k supervised ViT (ablation)iWildCam, Camelyon · iWildCam OODavg per-class accuracy: 18.57 -> 16.90Hurt
add WiSE-FT on CLIPCLIP ViTB-16 · ImageNet, iWildCam, Camelyon (all regimes)effective and relative robustnessImproved
add robustness interventions (LP-FT, WiSE-FT, Soups) on MSN ViTB-16MSN ViTB-16 · iWildCameffective/relative robustnessNo clear effect
add robustness interventions on MSN/CLIP ViTB-16 in low-shot regimesViTB-16 · Camelyon extreme and moderate low-shotrelative robustnessHurt
add RobustViT and other interventionsMSN ViTB-16 · ImageNeteffective and relative robustnessImproved
set WiSE-FT alpha to 1 instead of 0.5 with CLIP (ablation)WiSE-FT + CLIP · Camelyon full-shot OODavg per-class accuracy: 70.60 -> 94.27Improved
use Baseline++ instead of logistic regression classifier (ablation)MSN ViTS-16 · Camelyon low-shot OODavg per-class accuracy: 69.73 -> 75.66Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy