Research map / Architectures and efficiency / Attention and transformers
One Meta-tuned Transformer is What You Need for Few-shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MetaFormer, a ViT-based meta-tuned transformer for few-shot image classification that adds masked sample attention and patch-grained task attention to model sample and task relations, supporting both inductive and transductive protocols.
Problem
Few-shot image classification with pre-trained vision transformers; existing meta-learning methods model sample, patch, and task relationships separately with CNNs and task embeddings that carry background noise.
Approach
Builds a holistic attention transformer where self-attention spans patches, samples (Masked Sample Attention with a causal support/query mask and label infusion), and tasks (Patch-grained Task Attention with a dynamic knowledge pool), decoupling patch and sample attention for efficiency.
Key results
- MetaFormer-I reaches 75.78% 1-shot and 90.02% 5-shot on miniImageNet, exceeding FewTURE by 7.76% and 5.51%.
- Cross-domain 1-shot results improve over FewTURE and PMF by up to 8.77% and 16.86%.
- MetaFormer-A (transductive) reaches 84.78% 1-shot on miniImageNet, and adapting CLIP lifts EuroSAT 1-shot accuracy to 88.83% from 48.73% zero-shot.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PTA to MSA (ablation) | MetaFormer-I · miniImageNet 1-shot | accuracy: 74.64 ± 0.76 -> 75.78 ± 0.71 | Helped |
| Add PTA to MSA (ablation) | MetaFormer-I · miniImageNet 5-shot | accuracy: 87.53 ± 0.47 -> 90.02 ± 0.44 | Helped |
| Add MSA to PTA (ablation) | MetaFormer-I · miniImageNet 1-shot | accuracy: 73.63 ± 0.75 -> 75.78 ± 0.71 | Helped |
| Add MSA to PTA (ablation) | MetaFormer-I · miniImageNet 5-shot | accuracy: 87.76 ± 0.52 -> 90.02 ± 0.44 | Helped |
| Use decoupled instead of joint sample-patch attention (ablation) | MetaFormer · miniImageNet 1-shot | accuracy: 73.35 ± 0.75 -> 73.21 ± 0.76 | No clear effect |
| Add temperature scaling to cosine distance (ablation) | MetaFormer · miniImageNet 1-shot | accuracy: 64.22 ± 0.83 -> 74.14 ± 0.73 | Helped |
| Add patch-wise distance to temperature-scaled cosine distance (ablation) | MetaFormer · miniImageNet 1-shot | accuracy: 74.14 ± 0.73 -> 75.78 ± 0.71 | Helped |
| Use patch-wise Mahalanobis distance instead of cosine distance (ablation) | MetaFormer · miniImageNet 1-shot | accuracy: 75.78 ± 0.71 vs 71.97 ± 0.76 | Hurt |
| Add class labels by summation (ablation) | MetaFormer · miniImageNet 1-shot | accuracy: 73.70 -> 74.64 | Helped |
| Add class labels by concatenation (ablation) | MetaFormer · miniImageNet 1-shot | accuracy: 73.70 -> 74.59 | Helped |
| Reshuffle query sample order at inference (ablation) | MetaFormer-A · CIFAR-FS 1-shot | accuracy: 85.76 ± 0.77 -> 88.34 ± 0.76 | Helped |
| Use MetaFormer-I instead of FewTURE | ViT-Small few-shot learning · miniImageNet 1-shot | accuracy: 68.02±0.88 -> 75.78±0.71 | Helped |
Surprising findings
- Naively increasing the number of adapted parameters in Tip-Adapter-F with more layers causes overfitting and drops accuracy (e.g., EuroSAT 72.01% to 51.95%).
Limitations
- Evaluated only on few-shot image classification; detection and segmentation left for future work.
- Autoregressive (transductive) inference relies on inference-time shuffling augmentation with multiple passes.
Experimental setup
Datasets: miniImageNet tieredImageNet CIFAR-FS FC-100 Meta-Dataset
Benchmarks: miniImageNet tieredImageNet CIFAR-FS FC-100 CUB Cars Places Plantae CropDisease EuroSAT ISIC ChestX
Metrics: 5-way 1-shot accuracy 5-way 5-shot accuracy GFLOPs inference time
Baselines: FewTURE PMF SMKD HCTransformers protoLP EASY FEAT DeepEMD Tip-Adapter Zero-shot CLIP
Builds on: Vision Transformer DINO self-supervised pre-training FewTURE PMF Divided space-time attention
Closest papers by meaning
- A Dual Attention Network with Semantic Embedding for Few-Shot Learning (AAAI 2019 · similarity 0.89)
- Task-Adaptive Prompted Transformer for Cross-Domain Few-Shot Learning (AAAI 2024 · similarity 0.89)
- Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference (CVPR 2022 · similarity 0.89)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.88)
- Strong Baselines for Parameter-Efficient Few-Shot Fine-Tuning (AAAI 2024 · similarity 0.87)
- TADFormer: Task-Adaptive Dynamic TransFormer for Efficient Multi-Task Learning (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.