Research map / Architectures and efficiency / RNNs and normalization
On the Use of Anchoring for Training Vision Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Systematically studies anchored training for vision models and proposes reference-masking regularization to fix shortcut learning with large reference sets, improving OOD generalization, calibration, and anomaly rejection on CIFAR and ImageNet.
Problem
Anchored training (reparameterizing inputs as reference and residual pairs) does not effectively exploit large reference-set diversity, leading to shortcut learning and limited generalization in vision models.
Approach
A reference-masking regularizer that zeroes out the reference for a fraction of batches and trains the model to output uniform predictions on those masked tuples, discouraging reliance on residuals alone.
Key results
- On CIFAR-100 with WRN 40-2, the proposed method improves CIFAR100-C accuracy over standard training by 1.55 to 5.52 points across severities.
- On ImageNet-1K with SWINv2-B, the proposed method improves ImageNet-C severity 5 accuracy from 41.87 to 45.19 and ImageNet-C-bar severity 5 from 53.75 to 58.77.
- ViTb16 with the proposed method gains +4.34% AUROC over the non-anchored variant on NINCO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add reference-masking regularization to vanilla anchored training | ResNet-18 · CIFAR-100C Sev. 5 | accuracy: 34.79 -> 38.66 | Helped |
| Add reference-masking regularization to standard training | ResNet-18 · CIFAR-10C Sev. 5 | accuracy: +8.54 | Helped |
| Add reference-masking regularization to standard training | SWINv2-B · ImageNet-C Sev. 5 | top-1 accuracy: +3.32 | Helped |
| Add reference-masking regularization to standard training | ViTb16 · NINCO | AUROC: +4.34% | Helped |
| Add reference-masking regularization to standard training | ViTb16 · ImageNet OOD datasets | Smoothed ECE: 0.109±0.037 -> 0.105±0.028 | Helped |
| Use proposed training protocol instead of standard training | SWINv2-T · ImageNet-1K val | top-1 accuracy: 82.07 -> 82.03 | Hurt |
| Set masking probability α to 0.25 instead of 0.0 (ablation) | Anchored ResNet-18 · CIFAR-100C | OOD accuracy: 51.01 -> 53.77 | Helped |
| Set masking probability α to 1.0 instead of 0.25 (ablation) | Anchored ResNet-18 · CIFAR-100C | OOD accuracy: 53.77 -> 35.40 | Hurt |
| Set masking probability α to 1.0 instead of 0.25 (ablation) | Anchored ResNet-18 · CIFAR-100 ID test | accuracy: 77.89 -> 57.90 | Hurt |
| Replace single random reference inference with K random references or BLT (ablation) | Anchored CIFAR-100 model · CIFAR-100C | accuracy | No clear effect |
Surprising findings
- Vanilla anchored training with large reference sets (R=D) barely improves over reference set size, and more sophisticated inference protocols (K-random, bilinear transduction) give no significant accuracy gains over a single random reference.
- On ImageNet-1K, the proposed regularizer slightly reduces ID accuracy for SWINv2-T and ViTb16 (e.g., -0.31 for ViTb16) while improving OOD metrics.
Limitations
- No theoretical characterization of generalization for anchored models
- Gains are smaller or absent on some ImageNet ID and anomaly metrics (e.g., Textures and iSUN for some architectures)
- Hyperparameter alpha must be tuned per dataset and task (0.2 to 0.4), with high alpha causing underfitting
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K DomainNet CIFAR10-C CIFAR100-C ImageNet-C
Benchmarks: CIFAR10-C CIFAR10-C-bar CIFAR100-C CIFAR100-C-bar ImageNet-C ImageNet-C-bar ImageNet-R ImageNet-S iSUN Textures Places365 NINCO
Metrics: top-1 accuracy smoothed ECE AUROC
Baselines: Standard training Vanilla anchoring AugMix PixMix K random reference marginalization Bilinear transduction
Builds on: anchoring bilinear transduction AugMix PixMix
Closest papers by meaning
- Region Proposal by Guided Anchoring (CVPR 2019 · similarity 0.86)
- Train longer, generalize better: closing the generalization gap in large batch training of neural networks (NeurIPS 2017 · similarity 0.85)
- Anchor-based Robust Finetuning of Vision-Language Models (CVPR 2024 · similarity 0.84)
- Explicitly Imposing Constraints in Deep Networks via Conditional Gradients Gives Improved Generalization and Faster Convergence (AAAI 2019 · similarity 0.84)
- Beyond Random Augmentations: Pretraining with Hard Views (ICLR 2025 · similarity 0.83)
- Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.