Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Boosting Few-Shot Learning via Attentive Feature Regularization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Attentive Feature Regularization (AFR), which selects semantically related base categories via word2vec label similarity and applies instance- and channel-level attention to regularize frozen pre-trained features for few-shot classification.
Problem
Manifold regularization methods (Mixup, CutMix, PatchMix) mix randomly chosen, unrelated samples, which weakens feature representations and biases predictions in few-shot learning.
Approach
Select base categories by semantic (word2vec label) similarity to each novel category, then use self-attention calibration of their prototypes (instance level) and SE-style channel reweighting, training only a lightweight classifier on frozen features with CE, supervised contrastive and MSE losses.
Key results
- On Mini-ImageNet 1-shot, Meta-Baseline + AFR reaches 69.78% vs 63.17% for Meta-Baseline, and Label-Halluc + AFR reaches 74.57%, the best reported 1-shot accuracy in the table.
- Ablation on Mini-ImageNet 1-shot: instance attention plus channel attention raises accuracy from 64.02% to 70.68% with CE loss only; adding SC and MSE losses yields 72.35%.
- On Meta-Dataset with K=1, AFR achieves the best accuracy on all 8 test sets, e.g., 60.12% on Traffic Signs vs 56.65% for TCPR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add instance attention (ablation) | AFR · Mini-ImageNet, 5-way 1-shot | accuracy: 64.02 ± 0.70% -> 68.03 ± 0.58% | Helped |
| add channel attention (ablation) | AFR · Mini-ImageNet, 5-way 1-shot | accuracy: 64.02 ± 0.70% -> 68.74 ± 0.61% | Helped |
| add channel attention to instance attention (ablation) | AFR · Mini-ImageNet, 5-way 1-shot | accuracy: 68.03 ± 0.58% -> 70.68 ± 0.61% | Helped |
| add channel attention to instance attention (ablation) | AFR · Mini-ImageNet, 5-way 5-shot | accuracy: 83.04 ± 0.43% -> 83.36 ± 0.45% | Helped |
| add MSE loss (ablation) | AFR with instance and channel attention · Mini-ImageNet, 5-way 1-shot | accuracy: 70.68 ± 0.61% -> 70.93 ± 0.66% | Helped |
| add MSE loss (ablation) | AFR with instance and channel attention · Mini-ImageNet, 5-way 5-shot | accuracy: 83.36 ± 0.45% -> 83.76 ± 0.43% | Helped |
| add supervised contrastive loss (ablation) | AFR with instance and channel attention · Mini-ImageNet, 5-way 1-shot | accuracy: 70.68 ± 0.61% -> 71.18 ± 0.63% | Helped |
| add supervised contrastive loss (ablation) | AFR with instance and channel attention · Mini-ImageNet, 5-way 5-shot | accuracy: 83.36 ± 0.45% -> 83.70 ± 0.45% | Helped |
| add both supervised contrastive and MSE losses (ablation) | AFR with instance and channel attention · Mini-ImageNet, 5-way 1-shot | accuracy: 70.68 ± 0.61% -> 72.35 ± 0.63% | Helped |
| add both supervised contrastive and MSE losses (ablation) | AFR with instance and channel attention · Mini-ImageNet, 5-way 5-shot | accuracy: 83.36 ± 0.45% -> 84.11 ± 0.43% | Helped |
| use semantic category selection and instance attention instead of no regularization (ablation) | AFR · Mini-ImageNet validation, 5-way 1-shot and 5-shot | accuracy | Helped |
| select more than the most useful related categories (ablation) | AFR semantic selection · Mini-ImageNet validation, 5-way 1-shot and 5-shot | accuracy | Mixed |
Surprising findings
- Common regularizers (CutMix, Mixup, CutOut) give slight 1-shot gains over the baseline but reduce 5-shot accuracy on Mini-ImageNet in the authors' implementation.
Limitations
- Semantic relations are based only on word2vec label similarity, which the authors describe as superficial.
- Gains on Tiered-ImageNet are smaller for some integrations (e.g., FEAT + AFR 71.55% vs 70.80% for K=1 shows slight change, and FEAT's K=5 result is comparable), and per-method gains vary.
Experimental setup
Datasets: Mini-ImageNet Tiered-ImageNet Meta-Dataset CUB-200-2011 Omniglot Describable Textures Quick Draw Fungi VGG Flower Traffic Signs
Benchmarks: Mini-ImageNet Tiered-ImageNet Meta-Dataset
Metrics: accuracy (%)
Baselines: MatchingNets ProtoNets MixtFSL RENet DeepBDC FeLMi tSF RankDNN FRN BML FEAT Label-Halluc
Builds on: Manifold Mixup CutMix SE-Net FEAT Meta-Baseline
Closest papers by meaning
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.89)
- Boosting Few-Shot Learning With Adaptive Margin Loss (CVPR 2020 · similarity 0.88)
- Integrative Few-Shot Learning for Classification and Segmentation (CVPR 2022 · similarity 0.87)
- Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning (CVPR 2025 · similarity 0.86)
- Making Large Vision Language Models to Be Good Few-Shot Learners (AAAI 2025 · similarity 0.86)
- Multi-Label Few-Shot Image Classification via Pairwise Feature Augmentation and Flexible Prompt Learning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.