Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Frozen Feature Augmentation for Few-Shot Image Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies twenty image augmentations directly to cached frozen ViT features (FroFA) before training a lightweight MAP head, with brightness-style pointwise augmentations giving consistent few-shot gains.
Problem
Frozen features from pretrained vision models are used unmodified when training lightweight heads for few-shot classification, leaving out the data augmentation that helps when training on images directly.
Approach
Map image augmentations to frozen transformer features by reshaping tokens into a 2D grid and rescaling values to [0,1] before augmenting and mapping back, with per-channel variants (cFroFA, c2FroFA) adding stochasticity.
Key results
- Brightness c2FroFA on ILSVRC-2012 1-shot improves MAP baseline by 6.1% absolute, and brightness cFroFA reaches +1.6% on 5-shot.
- Averaged over seven few-shot datasets with JFT-3B L/16, MAPwd+FroFA beats MAPwd by 3.9%, 1.5%, 2.6%, 0.8% on 1/5/10/25-shot and linear probe by at least 4.0% absolute.
- With WebLI SigLIP L/16, MAPwd+FroFA gains 2.2% over the best baseline on 1-shot and 0.9% to 2.7% on 5- to 25-shot.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add brightness c2FroFA to MAPwd | JFT-3B L/16 · Seven few-shot datasets, 1-, 5-, 10-, and 25-shot mean | top-1 accuracy: 49.5 -> 53.4; 65.8 -> 67.3; 68.3 -> 70.9; 74.1 -> 74.9 | Helped |
| Add brightness c2FroFA to MAPwd | WebLI-SigLIP L/16 · Seven few-shot datasets, 1-, 5-, 10-, and 25-shot mean | top-1 accuracy: 45.9 -> 51.3; 67.7 -> 70.4; 71.8 -> 73.5; 75.1 -> 76.0 | Helped |
| Add brightness FroFA (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 1-, 5-, and 10-shot | top-1 accuracy: +4.8; +1.1; +0.6 | Helped |
| Add geometric frozen-feature augmentations (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 few-shot | top-1 accuracy: −1.3; −0.6; −0.8; −1.2; −1.4 | Hurt |
| Add crop FroFA (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 1-shot | top-1 accuracy: +3.0 | Helped |
| Add contrast FroFA (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 1-shot | top-1 accuracy: +2.8 | Helped |
| Add mixup FroFA (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 few-shot | top-1 accuracy: −1.4; −0.3; +0.2; +0.1 | Mixed |
| Apply brightness augmentation independently per channel (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 few-shot | top-1 accuracy: +5.9 vs +4.8; +1.5 vs +1.1; +1.1 vs +0.6; +0.4 vs +0.1 | Helped |
| Use per-channel feature-to-image mappings with brightness cFroFA (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 5-shot | top-1 accuracy: +1.6 | Helped |
| Apply contrast augmentation independently per channel (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 few-shot | top-1 accuracy: +2.8 vs +2.5; +0.8 vs +0.0; +0.6 vs +0.0; +0.1 vs −0.1 | Hurt |
| Add posterize cFroFA (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 1-shot | top-1 accuracy: +5.9 vs +3.7 | Helped |
| Apply brightness c2FroFA followed by posterize cFroFA (ablation) | JFT-3B L/16 MAP · ILSVRC-2012 1-shot | top-1 accuracy: +6.1 -> +7.7 | Helped |
Surprising findings
- Geometric augmentations (rotate, shear, translate) consistently hurt performance on ILSVRC-2012 frozen features.
- Per-element FroFA led to significantly worse results than per-channel or whole-feature variants.
Limitations
- Only ViT-Ti, B, and L sizes tested; larger models deferred to future work
- Sequential augmentation protocols and RandAugment/TrivialAugment variants showed limited success
- Gains on ILSVRC-2012 are modest at higher shots
Experimental setup
Datasets: ILSVRC-2012 CIFAR10 CIFAR100 DMLab DTD Resisc45 SUN397 SVHN JFT-3B ImageNet-21k WebLI
Benchmarks: ILSVRC-2012 CIFAR10 CIFAR100 DMLab DTD Resisc45 SUN397 SVHN
Metrics: top-1 accuracy absolute gain two-tailed t-test
Baselines: MAP MAPwd linear probe CLIP-Adapter TiP-Adapter
Builds on: AutoAugment RandAugment TrivialAugment linear probing Scenic
Closest papers by meaning
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.86)
- Multi-Label Few-Shot Image Classification via Pairwise Feature Augmentation and Flexible Prompt Learning (AAAI 2025 · similarity 0.86)
- Data-Free Reservoir Features for Efficient Long-Horizon Cold-Start Continual Learning (NeurIPS 2026 · similarity 0.85)
- Making Large Vision Language Models to Be Good Few-Shot Learners (AAAI 2025 · similarity 0.85)
- Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages (CVPR 2025 · similarity 0.85)
- Few-Class Arena: A Benchmark for Efficient Selection of Vision Models and Dataset Difficulty Measurement (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.