Research map / Detection and segmentation / Open-set and few-shot detection
Bridge the Points: Graph-based Few-shot Segment Anything Semantically
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GF-SAM, a training-free, hyperparameter-free SAM-based few-shot semantic segmentation method that selects point prompts via positive-negative alignment and clusters them with a directed coverage graph, using SAM only once.
Problem
Few-shot semantic segmentation with SAM requires selecting good point prompts and handling ambiguous SAM masks, but prior SAM-based FSS methods rely on many hyperparameters, repeated mask generation, and have slow inference.
Approach
Builds a directed graph whose vertices are point prompts and their SAM masks, with edges from mask coverage over other points; weakly connected components give parameter-free clusters used for positive gating (mask growth) and overshooting gating via self-consistency, with a single SAM mask-generation pass.
Key results
- Achieves 58.7% mIoU on COCO-20i 1-shot and 35.2% on LVIS-92i 1-shot, surpassing Matcher (52.7%, 33.0%) among generalist models.
- Reaches 44.5% on PASCAL-Part and 36.3% on PACO-Part one-shot part segmentation, +1.6 over Matcher on both.
- Scores 49.5% on Deepglobe, 48.7% on ISIC and 47.1% on iSAID-5i 1-shot, state-of-the-art on cross-domain FSS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bridge the Points instead of Matcher | SAM-based few-shot segmentation · COCO-20i 1-shot | mIoU: 52.7 -> 58.7 | Helped |
| Use Bridge the Points instead of Matcher | SAM-based few-shot segmentation · LVIS-92i 1-shot | mIoU: 33.0 -> 35.2 | Helped |
| Use Bridge the Points instead of Matcher | One-shot part segmentation · PASCAL-Part | mIoU: +1.6% | Helped |
| Use Bridge the Points instead of Matcher | Cross-domain few-shot segmentation · ISIC 1-shot | mIoU: 38.6 -> 48.7 | Helped |
| Omit S+max from point selection (ablation) | Bridge the Points · COCO-20i 1-shot | mIoU: 58.7 -> 53.1 | Hurt |
| Omit S+mean from point selection (ablation) | Bridge the Points · COCO-20i 1-shot | mIoU: 58.7 -> 54.1 | Hurt |
| Omit Top N point selection (ablation) | Bridge the Points · COCO-20i 1-shot | mIoU: 56.4 -> 58.7 | Hurt |
| Omit negative similarity from point selection (ablation) | Bridge the Points · COCO-20i 1-shot | mIoU: 51.5 -> 58.7 | Hurt |
| Remove clustering-based gating (ablation) | Bridge the Points · COCO-20i 1-shot | mIoU: 44.0 -> 58.7 | Hurt |
| Replace weakly connected components with k-means++ (ablation) | Bridge the Points · COCO-20i 1-shot | mIoU: 57.5 -> 58.7 | Hurt |
| Replace weakly connected components with k-means++ (ablation) | Bridge the Points · LVIS-92i 1-shot | mIoU: 34.0 -> 35.2 | Hurt |
| Use Mask Growth rather than judging masks separately (ablation) | Positive Gating · COCO-20i | mIoU: 57.1 -> 58.7 | Helped |
Surprising findings
- Using the mean of positive similarity alone or max alone drops performance by up to 5.6% versus combining them, showing the sensitivity of point selection design.
- Replacing the hyperparameter-free weakly connected component clustering with k-means++ (k=10) reduces COCO-20i mIoU from 58.7 to 57.5.
Limitations
- Coordinate mapping from DINOv2 feature resolution to SAM prompts causes coordinate bias for small objects (gaps up to ~28 pixels).
- Clustering relies on high-quality SAM masks whose boundaries align with objects; errors can hurt clustering accuracy.
- Positive gating and overshooting gating can reduce performance on some tasks (e.g., Union strategy performs worse on part segmentation).
Experimental setup
Datasets: Pascal-5i COCO-20i FSS-1000 LVIS-92i PASCAL-Part PACO-Part Deepglobe ISIC2018 iSAID-5i
Benchmarks: Pascal-5i COCO-20i FSS-1000 LVIS-92i PASCAL-Part PACO-Part Deepglobe ISIC2018 iSAID-5i
Metrics: mIoU
Baselines: HSNet VAT HDMNet AMFormer PerSAM PerSAM-F Matcher VRP-SAM DRA FRINet
Builds on: Segment Anything Model DINOv2 Matcher PerSAM Masked Average Pooling
Closest papers by meaning
- SAM-Aware Graph Prompt Reasoning Network for Cross-Domain Few-Shot Segmentation (AAAI 2025 · similarity 0.91)
- SANSA: Unleashing the Hidden Semantics in SAM2 for Few-Shot Segmentation (NeurIPS 2025 · similarity 0.90)
- Semantic-aware SAM for Point-Prompted Instance Segmentation (CVPR 2024 · similarity 0.90)
- From SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.89)
- APSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation (CVPR 2024 · similarity 0.89)
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.