Research map / Detection and segmentation / Open-set and few-shot detection
CFR-ICL: Cascade-Forward Refinement with Iterative Click Loss for Interactive Image Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CFR-ICL, a click-based interactive segmentation framework with Cascade-Forward Refinement inference, an Iterative Click Loss that penalizes click count, and SUEM copy-paste augmentation, built on SimpleClick ViT models.
Problem
Click-based interactive segmentation relies on heuristic, separately trained refinement modules and cannot balance the number of user clicks against segmentation quality.
Approach
Train with a loss summed over the sequence of iteratively generated click-conditioned masks (weighted by beta_i) so the model is rewarded for accurate masks with fewer clicks; at inference, refine each mask multiple times with the same clicks (CFR) without an extra network.
Key results
- On the C+L-trained ViT-H setting, Ours CFR-1 reduces NoC@95 on Berkeley by 33.2% (2.90 vs 4.34 baseline) and on DAVIS NoC@95 by 15.6% (7.50 vs 8.88).
- CFR-1 on SBD-trained SimpleClick ViT-H reduces NoC@95 on DAVIS by 24.2% relative to its baseline (8.85 vs 11.67) and NoC@90 on Berkeley by 16.7%.
- On HRNet32, ICL+SUEM C&P reduces NoC@95 vs RITM on GrabCut by 20.97%, Berkeley by 6.65%, and DAVIS by 5.82%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ICL during SimpleClick fine-tuning (ablation) | SimpleClick ViT-Base · GrabCut | NoC@95: 2.16 -> 2.00 | Helped |
| Add ICL during SimpleClick fine-tuning (ablation) | SimpleClick ViT-Base · Berkeley | NoC@95: 6.71 -> 6.48 | Helped |
| Add SUEM C&P during SimpleClick fine-tuning (ablation) | SimpleClick ViT-Base · GrabCut, Berkeley, and DAVIS | NoC@95: 2.16 -> 1.74; 6.71 -> 5.08; 12.23 -> 10.69 | Helped |
| Add SUEM C&P during SimpleClick fine-tuning (ablation) | SimpleClick ViT-Base · Pascal VOC and SBD | NoC@95: 3.75 -> 3.59; 11.23 -> 11.48 | Hurt |
| Apply one CFR refinement step at inference (ablation) | SimpleClick ViT-Base · SBD test set, all five datasets in CFR table | NoC: StdInfer: 1.54, 2.16, 2.46, 6.71, 5.48, 12.23, 2.81, 3.75, 5.24, 11.23; CFR-1: 1.44, 2.04, 2.35, 6.43, 5.33, 11.99, 2.70 | Helped |
| Increase fixed CFR refinement steps from one to four (ablation) | SimpleClick ViT-Base with CFR · GrabCut, Berkeley, DAVIS, Pascal VOC, and SBD | NoC: CFR-1: 1.44, 2.04, 2.35, 6.43, 5.33, 11.99, 2.70, 3.58, 5.11, 11.14; CFR-4: 1.50, 2.18, 2.38, 6.46, 5.31, 11.94, 2.72, 3 | Mixed |
| Use adaptive CFR with maximum four steps instead of fixed four-step CFR (ablation) | SimpleClick ViT-Base with CFR · GrabCut, Berkeley, DAVIS, Pascal VOC, and SBD | NoC: CFR-4: 1.50, 2.18, 2.38, 6.46, 5.31, 11.94, 2.72, 3.59, 5.16, 11.16; A-CFR-4: 1.50, 2.06, 2.39, 6.43, 5.28, 11.90, 2.73, | Mixed |
| Add ICL and SUEM C&P to SimpleClick and use CFR-1 inference | SimpleClick ViT-H trained on C+L · Berkeley | NoC@95: 4.34 -> 2.90 | Helped |
| Add ICL and SUEM C&P to SimpleClick and use CFR-1 inference | SimpleClick ViT-H trained on C+L · DAVIS | NoC@95: 8.88 -> 7.50 | Helped |
| Add ICL and SUEM C&P to SimpleClick and use CFR-1 inference | SimpleClick ViT-H trained on SBD · GrabCut, Pascal VOC, SBD, Berkeley, and DAVIS | NoC@95: 2.10 -> 1.62; 3.01 -> 2.91; 9.86 -> 10.50; 6.34 -> 4.44; 11.67 -> 8.85 | Mixed |
Surprising findings
- Increasing CFR refinement steps from 1 to 4 (CFR-4) does not reliably improve over CFR-1, suggesting more steps do not guarantee better outcomes.
- CFR gives no notable improvement on GrabCut, Pascal VOC, and SBD, which the authors attribute to simple visual content and inaccurate SBD annotations.
Limitations
- Improvements on GrabCut, Pascal VOC, and SBD are not significant, attributed to simple content and noisy SBD annotations.
- Fine-tuning gains from ICL on SimpleClick are modest in several settings (e.g., NoC@95 on SBD slightly worse for SUEM C&P).
Experimental setup
Datasets: SBD COCO LVIS C+L Pascal VOC SA-1B
Benchmarks: GrabCut Berkeley DAVIS Pascal VOC SBD
Metrics: NoC@90 NoC@95
Baselines: SimpleClick RITM EMC-Click FocalClick f-BRS SAM PseudoClick CDNet
Builds on: SimpleClick RITM Simple Copy-Paste Normalized Focal Loss
Closest papers by meaning
- Variance-Insensitive and Target-Preserving Mask Refinement for Interactive Image Segmentation (AAAI 2024 · similarity 0.87)
- NTClick: Achieving Precise Interactive Segmentation With Noise-tolerant Clicks (CVPR 2025 · similarity 0.86)
- TETRIS: Towards Exploring the Robustness of Interactive Segmentation (AAAI 2024 · similarity 0.86)
- RClicks: Realistic Click Simulation for Benchmarking Interactive Segmentation (NeurIPS 2024 · similarity 0.86)
- FocSAM: Delving Deeply into Focused Objects in Segmenting Anything (CVPR 2024 · similarity 0.85)
- Order-aware Interactive Segmentation (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.