Research map / Detection and segmentation / Open-set and few-shot detection
Foreground-Covering Prototype Generation and Matching for SAM-Aided Few-Shot Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FCP, a SAM-aided few-shot segmentation method that builds support and query prototypes from SAM and ResNet features and matches them to produce visual reference prompts for the SAM mask decoder.
Problem
Few-shot segmentation: segmenting novel-class target regions in query images given a few labeled support images, where SAM lacks class consistency and prior pixel-to-prototype prompt matching (VRP-SAM) gives poor foreground coverage.
Approach
Instead of matching support prototypes against query pixels, build foreground-covering prototypes for both support and query: SAM features aggregate foreground via iterative masked cross-attention into learnable tokens, ResNet features inject class consistency, and the query's attention-based pseudo-mask replaces the conventional pseudo-mask; prototype-to-prototype matching yields prompts for the
Key results
- On PASCAL-5i 1-shot with ResNet-50, FCP reaches 73.2 mean mIoU versus 71.8 for VRP-SAM (reproduced).
- On COCO-20i 1-shot with ResNet-50, FCP reaches 52.5 mean mIoU versus 50.2 for VRP-SAM (reproduced).
- Attention-based pseudo-mask raises pseudo-mask mIoU from 32.4 to 60.9 compared with the conventional pseudo-mask.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FCP instead of VRP-SAM with VGG-16 | FSS · PASCAL-5^i 1-shot | mIoU: 68.4 -> 70.5 | Helped |
| Use FCP instead of the strongest listed baseline with VGG-16 | FSS · COCO-20^i 5-shot | mIoU: 56.0 -> 53.7 | Hurt |
| Use FCP instead of VRP-SAM with ResNet-50 | FSS · PASCAL-5^i 1-shot | mIoU: 71.8 -> 73.2 | Helped |
| Use FCP instead of the strongest listed baseline with ResNet-50 | FSS · COCO-20^i 5-shot | mIoU: 56.0 -> 58.0 | Helped |
| Use SAM features instead of ResNet features in the prototype-pixel baseline (ablation) | FSS prototype construction · PASCAL-5^i, ResNet-50 | mIoU: 71.8 -> 66.2 | Hurt |
| Add prototype-prototype matching to ResNet-only features (ablation) | FSS prototype construction · PASCAL-5^i, ResNet-50 | mIoU: 71.8 -> 66.5 | Hurt |
| Add prototype-prototype matching to SAM-only features (ablation) | FSS prototype construction · PASCAL-5^i, ResNet-50 | mIoU: 66.2 -> 66.1 | No clear effect |
| Combine ResNet and SAM features with prototype-prototype matching (ablation) | FSS prototype construction · PASCAL-5^i, ResNet-50 | mIoU: 71.8 -> 72.6 | Helped |
| Add the attention-based pseudo-mask to combined-feature prototype-prototype matching (ablation) | FCP · PASCAL-5^i, ResNet-50 | mIoU: 72.6 -> 73.2 | Helped |
| Replace the conventional pseudo-mask with the attention-based pseudo-mask (ablation) | Pseudo-mask generation · Pseudo-mask comparison | mIoU: 32.4 -> 60.9 | Helped |
| Add guide loss to prompt loss (ablation) | FCP training · PASCAL-5^i, ResNet-50 | mIoU: 72.3 -> 72.7 | Helped |
| Add orthogonal loss to prompt loss (ablation) | FCP training · PASCAL-5^i, ResNet-50 | mIoU: 72.3 -> 72.4 | Helped |
Surprising findings
- Using SAM features alone instead of ResNet features for prototype matching drops performance sharply (66.2 vs 71.8 mIoU), and prototype-to-prototype matching with ResNet alone slightly hurts (66.5).
Limitations
- Some query prototypes may encode background because no query ground-truth mask guides their construction.
- Performance on the attention-based pseudo-mask shrinks with too many aggregation steps (T), requiring tuning of T.
Experimental setup
Datasets: PASCAL-5i COCO-20i
Benchmarks: PASCAL-5i COCO-20i
Metrics: mIoU
Baselines: HDMNet VRP-SAM PFENet CyCTR SSP BAM
Builds on: VRP-SAM SAM PFENet
Closest papers by meaning
- Unlocking the Power of SAM 2 for Few-Shot Segmentation (ICML 2025 · similarity 0.90)
- Task-Disruptive Background Suppression for Few-Shot Segmentation (AAAI 2024 · similarity 0.89)
- Adaptive FSS: A Novel Few-Shot Segmentation Framework via Prototype Enhancement (AAAI 2024 · similarity 0.88)
- Addressing Background Context Bias in Few-Shot Segmentation through Iterative Modulation (CVPR 2024 · similarity 0.88)
- CGMGM: A Cross-Gaussian Mixture Generative Model for Few-Shot Semantic Segmentation (AAAI 2024 · similarity 0.87)
- DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.