Research map / Detection and segmentation / Open-set and few-shot detection
SAUGE: Taming SAM for Uncertainty-Aligned Multi-Granularity Edge Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAUGE, a lightweight side transfer network injected into a frozen SAM that fuses intermediate features into coarse-to-fine edge outputs trained on linearly blended pseudo labels, enabling controllable granularity edge detection.
Problem
Edge detection labels are subjective and vary in granularity across annotators, causing uncertainty that prior methods handle with voting or fixed distributional assumptions, lacking granularity control.
Approach
Align label uncertainty with granularity by regressing intermediate SAM features from different layers to multi-granularity edge pseudo labels (created by OR-combining sorted annotations), with a frozen SAM plus ~1.5% extra parameters, and generate arbitrary-granularity edges by linear blending of side outputs.
Key results
- On BSDS500 (SS setting), SAUGE with 1.3M params reaches ODS 0.839 / OIS 0.860 / AP 0.893, and SAUGE-L reaches ODS 0.847 / OIS 0.868 / AP 0.898, beating EdgeSAM (ODS 0.838, OIS 0.852).
- On Multicue, SAUGE achieves ODS 0.905 (best) and OIS 0.907 (second), with AP 0.939 below MuGE's 0.950.
- Zero-shot on NYUDv2, SAUGE-L* reaches ODS 0.794 and OIS 0.803, outperforming supervised EdgeSAM and RankED on ODS/OIS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the zero-shot SAM baseline with SAUGE's Side Transfer Network (STN) (ablation) | SAM ViT-B · BSDS500 | ODS: .768 -> .837 | Helped |
| Add side-output constraints (SOC) to SAUGE (ablation) | SAUGE · BSDS500 | AP: .880 -> .891 | Helped |
| Add mask-guided loss to SAUGE with SOC (ablation) | SAUGE · BSDS500 | ODS: .836 -> .839 | Helped |
| Add mask-guided loss to SAUGE with SOC (ablation) | SAUGE · BSDS500 | OIS: .857 -> .860 | Helped |
| Add mask-guided loss to SAUGE with SOC (ablation) | SAUGE · BSDS500 | AP: .891 -> .893 | Helped |
| Add side-output constraints (SOC) to SAUGE-L (ablation) | SAUGE-L · BSDS500 | AP: .822 -> .895 | Helped |
| Add mask-guided loss to SAUGE-L with SOC (ablation) | SAUGE-L · BSDS500 | ODS: .846 -> .847 | Helped |
| Add mask-guided loss to SAUGE-L with SOC (ablation) | SAUGE-L · BSDS500 | OIS: .866 -> .868 | Helped |
| Add mask-guided loss to SAUGE-L with SOC (ablation) | SAUGE-L · BSDS500 | AP: .895 -> .898 | Helped |
| Use ViT-L instead of ViT-B SAM as the backbone | SAUGE · BSDS500 test, SS | ODS: +0.009 | Helped |
| Increase multi-granularity candidates from M=3 to M=11 | SAUGE · BSDS500 test, SS | ODS: .854 -> .857 | Helped |
| Use SAUGE instead of MuGE for multi-granularity edge detection | SAUGE and MuGE · BSDS500 test, SS, M=3 | ODS: .845 -> .854 | Helped |
Surprising findings
- Zero-shot SAUGE-L transferred from BSDS500 and PASCAL VOC outperforms several supervised methods on NYUDv2 in ODS and OIS.
- On Multicue, SAUGE is lower than MuGE in AP, which the authors attribute to overfitting on the small dataset.
Limitations
- Built entirely on the SAM backbone, which is large and inefficient; the authors plan a more efficient backbone.
- Multicue AP is lower than prior work, possibly due to overfitting on the small dataset.
Experimental setup
Datasets: BSDS500 PASCAL VOC Multicue NYUDv2 SA-1B
Benchmarks: BSDS500 Multicue NYUDv2
Metrics: ODS OIS AP
Baselines: Canny OEF DeepContour DeepBoundary HED RCF CED BDCN EDTER UAED RankED DiffusionEdge
Builds on: Segment Anything Model EdgeSAM MuGE UAED RankED
Closest papers by meaning
- Uncertainty-aware Fine-tuning of Segmentation Foundation Models (NeurIPS 2024 · similarity 0.88)
- RankED: Addressing Imbalance and Uncertainty in Edge Detection Using Ranking-based Losses (CVPR 2024 · similarity 0.87)
- DiffusionEdge: Diffusion Probabilistic Model for Crisp Edge Detection (AAAI 2024 · similarity 0.85)
- Flaws can be Applause: Unleashing Potential of Segmenting Ambiguous Objects in SAM (NeurIPS 2024 · similarity 0.85)
- UNICL-SAM: Uncertainty-Driven In-Context Segmentation with Part Prototype Discovery (CVPR 2025 · similarity 0.85)
- MuGE: Multiple Granularity Edge Detection (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.