Research map / Detection and segmentation / Open-set and few-shot detection
FocSAM: Delving Deeply into Focused Objects in Segmenting Anything
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FocSAM, which adds a lightweight focus refiner (Dwin-MSA plus P-DyReLU) to SAM's pipeline to refocus image embeddings on the target object and fuse early clicks, improving interactive segmentation stability and matching SimpleClick NoC at about 5.6% of its CPU time.
Problem
SAM's interactive segmentation is unstable on challenging samples because its precomputed image embeddings cannot be zoomed into the target object and its lightweight decoder fuses clicks weakly.
Approach
A focus refiner run once per object refines SAM's image embeddings: Dynamic Window MSA attends only to windows intersecting the current mask's bounding box, and Pixel-wise Dynamic ReLU uses click-fused query embeddings to modulate activations pixel-wise.
Key results
- FocSAM-ViT-H achieves mean NoC@90 of 5.38 across six datasets, the best among compared methods (SimpleClick-ViT-H 5.40), with 0.39 s/click CPU vs 6.99 for SimpleClick-ViT-H.
- On SBD, FocSAM reduces SAM's NoC@90 from 7.62 to 4.69 (20 clicks) and COD10K from 10.36 to 8.91.
- Stability analysis shows FocSAM has fewer significant IoU drops with additional clicks than SAM on SBD, MVTec and COD10K.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FocSAM's focus refiner to SAM | SAM ViT-H · GrabCut, Berkeley, SBD, DAVIS, and COD10K | NoC@90: GrabCut: 1.88 -> 1.32; Berkeley: 2.09 -> 1.47; SBD: 7.62 -> 4.69; DAVIS: 5.19 -> 4.77; COD10K: 10.36 -> 8.91 | Helped |
| Add FocSAM's focus refiner to SAM | SAM ViT-H · MVTec | NoC@90: 13.97 -> 11.14 | Helped |
| Use FocSAM instead of SimpleClick-ViT-H | Interactive segmentation CPU inference · CPU inference | SPC/s: 6.99 -> 0.39 | Helped |
| Add Dwin-MSA and P-DyReLU (ablation) | FocSAM · SBD, MVTec, and COD10K | 100NoC@95: SBD: 63.40 -> 32.96; MVTec: 81.90 -> 62.82; COD10K: 76.73 -> 62.61 | Helped |
| Add Dwin-MSA (ablation) | FocSAM without Dwin-MSA or P-DyReLU · SBD, MVTec, and COD10K | 20NoC@90: SBD: 7.62 -> 4.75; MVTec: 13.97 -> 11.29; COD10K: 10.36 -> 9.26 | Helped |
| Add P-DyReLU (ablation) | FocSAM without Dwin-MSA or P-DyReLU · SBD, MVTec, and COD10K | 20NoC@90: SBD: 7.62 -> 4.76; MVTec: 13.97 -> 11.48; COD10K: 10.36 -> 9.33 | Helped |
| Add P-DyReLU to Dwin-MSA (ablation) | FocSAM with Dwin-MSA only · SBD, MVTec, and COD10K | 100NoC@95: SBD: 34.39 -> 32.96; MVTec: 64.15 -> 62.82; COD10K: 64.32 -> 62.61 | Helped |
| Add Dwin-MSA to P-DyReLU (ablation) | FocSAM with P-DyReLU only · SBD, MVTec, and COD10K | 100NoC@95: SBD: 34.52 -> 32.96; MVTec: 65.04 -> 62.82; COD10K: 64.41 -> 62.61 | Helped |
Surprising findings
- Plain SAM with the proposed refiner matches or beats SimpleClick-ViT-H on NoC while being far faster on CPU, yet SAM alone is weaker than early methods like FocalClick.
Limitations
- Slightly underperforms SimpleClick on the MVTec dataset.
- Training depends on a two-stage schedule because the refiner loss depends on the decoder, causing instability otherwise.
Experimental setup
Datasets: COCO LVIS GrabCut Berkeley SBD DAVIS MVTec COD10K
Benchmarks: GrabCut Berkeley SBD DAVIS MVTec COD10K
Metrics: NoC@90 NoC@95 seconds per click (CPU) delta IoU stability
Baselines: SAM-ViT-H SimpleClick-ViT-H SimpleClick-ViT-L FocalClick InterFormer-Tiny RITM-HR32 f-BRS-B-HR32
Builds on: Segment Anything Model DyReLU SimpleClick InterFormer f-BRS
Closest papers by meaning
- SAM-REF: Introducing Image-Prompt Synergy during Interaction for Detail Enhancement in the Segment Anything Model (CVPR 2025 · similarity 0.90)
- Stable Segment Anything Model (ICLR 2025 · similarity 0.88)
- AoP-SAM: Automation of Prompts for Efficient Segmentation (AAAI 2025 · similarity 0.88)
- Rethinking Interactive Image Segmentation with Low Latency High Quality and Diverse Prompts (CVPR 2024 · similarity 0.88)
- Order-aware Interactive Segmentation (ICLR 2025 · similarity 0.88)
- Personalize Segment Anything Model with One Shot (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.