Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Multi-Label Few-Shot Image Classification via Pairwise Feature Augmentation and Flexible Prompt Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a prototype-based multi-label few-shot image classifier that augments region features of support-image pairs (PFA) and learns class-specific prompts (FPL) to build text prototypes, then adaptively merges them with image prototypes.
Problem
Multi-label few-shot image classification suffers from data scarcity and interference from irrelevant classes in multi-label images; prior work addresses only one of these issues.
Approach
Pairwise feature augmentation on CLIP region features (weighted by keys) generates extra image features for image prototypes, while conditional-layer-norm prompt pool produces class-specific prompts for text prototypes; learnable weights fuse both prototypes and uncertainty weighting balances losses.
Key results
- On COCO, the method reaches micro F1 73.80 and micro AP 79.36 versus NegP's 62.06 and 67.35 (improvements of 6.38 F1 and 12.01 AP).
- On VOC, micro AP 90.95 and macro AP 93.50 outperform the strongest baseline NegP (86.97 and 89.82).
- Removing text prototypes drops COCO miAP from 79.36 to 49.26, and removing flexible prompt learning drops it to 50.32.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add pairwise feature augmentation (ablation) | Ours · COCO | micro AP: 75.02 -> 79.36 | Helped |
| add pairwise feature augmentation (ablation) | Ours · COCO | macro AP: 78.84 -> 82.85 | Helped |
| add pairwise feature augmentation (ablation) | Ours · VOC | micro AP: 89.34 -> 90.95 | Helped |
| add pairwise feature augmentation (ablation) | Ours · VOC | macro AP: 92.10 -> 93.50 | Helped |
| use flexible prompts instead of class text features alone (ablation) | Ours · COCO | micro AP: 50.32 -> 79.36 | Helped |
| use flexible prompts instead of class text features alone (ablation) | Ours · COCO | macro AP: 80.50 -> 82.85 | Helped |
| use flexible prompts instead of class text features alone (ablation) | Ours · VOC | micro AP: 80.87 -> 90.95 | Helped |
| use flexible prompts instead of class text features alone (ablation) | Ours · VOC | macro AP: 92.33 -> 93.50 | Helped |
| combine image and text prototypes (ablation) | Ours · COCO | micro AP: 49.26 -> 79.36 | Helped |
| combine image and text prototypes (ablation) | Ours · VOC | micro AP: 70.26 -> 90.95 | Helped |
| use adaptive image/text prototype weighting rather than averaging (ablation) | Ours · COCO | micro AP: 76.38 -> 79.36 | Helped |
| use adaptive image/text prototype weighting rather than averaging (ablation) | Ours · VOC | micro AP: 87.19 -> 90.95 | Helped |
Surprising findings
- Using only text prototypes outperforms using only image prototypes on both datasets, suggesting text prototypes carry most of the discriminative signal.
Limitations
- Relies on frozen CLIP ResNet50 encoders and the WVAtten episode protocol, limiting generality to other backbones.
- Evaluated only on COCO and VOC with 1-shot episodes.
- Reported gains rely on grid-searched hyperparameters tuned on validation split.
Experimental setup
Datasets: MS COCO PASCAL VOC
Benchmarks: MS COCO PASCAL VOC
Metrics: micro precision micro recall micro F1 micro AP macro precision macro recall macro F1 macro AP
Baselines: WVAtten WVAtten* DualCoop SPM NegP
Builds on: CLIP DualCoOp WVAtten LaSO conditional layer normalization
Closest papers by meaning
- Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention (AAAI 2022 · similarity 0.91)
- Correlative and Discriminative Label Grouping for Multi-Label Visual Prompt Tuning (CVPR 2025 · similarity 0.88)
- Self-Prompt Mechanism for Few-Shot Image Recognition (AAAI 2024 · similarity 0.87)
- Text Augmented Correlation Transformer For Few-shot Classification & Segmentation (CVPR 2025 · similarity 0.86)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.86)
- Making Large Vision Language Models to Be Good Few-Shot Learners (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.