Research map / Detection and segmentation / Open-set and few-shot detection
Adaptive FSS: A Novel Few-Shot Segmentation Framework via Prototype Enhancement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adaptive FSS inserts a lightweight Prototype Adaptive Module (prototype enhancement plus a small bottleneck adapter) into frozen few-shot segmentation backbones and fine-tunes only that module on support samples before testing, improving MSANet, HDMNet, FPTrans and DCAMA on PASCAL-5i and COCO-20i.
Problem
Existing meta-learning few-shot segmentation methods rely on fixed query-support feature matching at test time and cannot adapt to novel classes, while standard fine-tuning of FSS models risks overfitting and catastrophic forgetting with very few labeled samples.
Approach
Insert a Prototype Adaptive Module (PAM) into the encoder of existing FSS models; PAM builds class prototypes from the support mask with a momentum-updated prototype bank, uses them to enhance class-specific features, and passes them through a small down/up projection adapter, while all other weights stay frozen.
Key results
- On PASCAL-5i, FPTrans + Ours reaches 72.3% mIoU 1-shot (+3.5 over reproduced FPTrans) with 0.45M added parameters, and DCAMA + Ours reaches 72.4% (+3.1).
- On COCO-20i, the method reaches 52.7% (1-shot) and 60.0% (5-shot) mIoU with FPTrans + Ours, and DCAMA + Ours gives 52.7% 1-shot (+1.8).
- Adaptation outperforms F-Head, F-Decoder, AdaptFormer and VPT across backbones; cross-domain COCO-to-PASCAL FPTrans improves from 68.2% to 72.5% mIoU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LAM to the unadapted FPTrans baseline (ablation) | FPTrans on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 68.8 -> 70.1 | Helped |
| Add PEM using a temporary prototype, without the class prototype bank (ablation) | FPTrans + LAM on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 70.1 -> 71.4 | Helped |
| Add the class prototype bank (CPB) (ablation) | FPTrans + LAM + PEM on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 71.4 -> 72.3 | Helped |
| Insert PAM in deeper layers rather than early layers (ablation) | FPTrans on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 71.6 -> 72.3 | Helped |
| Use momentum ratio α=0.99 instead of α=0.999 (ablation) | FPTrans + PAM on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 72.0 -> 72.3 | Helped |
| Use scaling factor β=0.1 rather than β=1 or β=0.5 (ablation) | FPTrans + PAM on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 66.7 -> 72.3; 69.2 -> 72.3 | Helped |
| Set hidden ratio γ=16 rather than γ=32 (ablation) | FPTrans + PAM on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 71.5 -> 72.3 | Helped |
| Add Adaptive FSS to DCAMA | DCAMA · PASCAL-5ᵢ 1-shot and 5-shot | mIoU: 69.3 -> 72.4; 74.9 -> 76.7 | Helped |
| Add Adaptive FSS to FPTrans | FPTrans · COCO-20ᵢ 1-shot and 5-shot | mIoU: 47.0 -> 48.3; 58.9 -> 60.0 | Helped |
| Fine-tune the decoder instead of using the unadapted MSANet | MSANet on COCO-20ᵢ 1-shot · COCO-20ᵢ | mIoU: 49.5 -> 49.2 | Hurt |
| Use PAM insertion in the last stage instead of all ResNet stages (ablation) | MSANet on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 67.9 -> 69.5 | Helped |
| Use only one labeled sample with data augmentation for support | FPTrans on PASCAL-5ᵢ 1-shot · PASCAL-5ᵢ | mIoU: 68.8 -> 70.4 | Helped |
Surprising findings
- Inserting PAM only in deep layers (last stage) works better than inserting in shallow or all layers, contrary to the usual assumption that more adaptation layers help.
- Fine-tuning only the classification head (F-Head, 290 params) gives almost no gain, suggesting that head-only adaptation is insufficient for dense FSS.
Limitations
- Gains on the 5-shot and domain-shift settings depend on the reproduced baselines, and some reported baseline numbers differ from the original papers.
- Fine-tuning relies on labeled support samples at test time, so it adds per-task optimization cost compared with pure feed-forward FSS.
- The single-sample evaluation uses augmented copies of the same image as query and support, which is a weaker setting.
Experimental setup
Datasets: PASCAL VOC 2012 SBD COCO 2014
Benchmarks: PASCAL-5i COCO-20i
Metrics: mIoU
Baselines: MSANet HDMNet FPTrans DCAMA BAM MIANet AdaptFormer VPT F-Decoder F-Head Conv-Adapter
Builds on: adapter tuning AdaptFormer VPT MSANet HDMNet FPTrans DCAMA
Closest papers by meaning
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.89)
- Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentation (CVPR 2024 · similarity 0.88)
- Foreground-Covering Prototype Generation and Matching for SAM-Aided Few-Shot Segmentation (AAAI 2025 · similarity 0.88)
- Unlocking the Power of SAM 2 for Few-Shot Segmentation (ICML 2025 · similarity 0.88)
- Text Augmented Correlation Transformer For Few-shot Classification & Segmentation (CVPR 2025 · similarity 0.88)
- DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.