Research map / Detection and segmentation / Open-set and few-shot detection
Variance-Insensitive and Target-Preserving Mask Refinement for Interactive Image Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes mask matching regularization and target-aware image zooming (TAIZ) to make point-based interactive segmentation robust to initial mask variance and preserve target detail during downsampling, achieving state-of-the-art NoC on GrabCut, Berkeley, SBD, and DAVIS.
Problem
Point-based interactive image segmentation needs many clicks for full mask extraction; iterative refinement models are sensitive to the initial mask, and standard downsampling (bilinear interpolation) loses target information.
Approach
A consistency regularization that forces matching predictions from two different initial-mask variants (a click-synthesized coarse mask and a GT-perturbed mask), plus a guidance-map-driven inverse mapping resampler (TAIZ) that allocates denser sampling to the target region during downsampling.
Key results
- With SegFormer-B3, NoC@85 on DAVIS is 3.26 versus 3.61 for FocalClick-segformerB3 (reduction of 0.35), and NoC@90 on Berkeley is 1.72 versus 1.92.
- Ablation on DAVIS: MaskMatch reduces NoC@85 by 0.07 and TAIZ by 0.13 relative to the baseline; combining both gives 3.26.
- Per-click inference time (SPC) is 0.054s on Berkeley and 0.067s on DAVIS with SegFormer-B3, described as congruent with real-time use.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MaskMatch to the baseline without MaskMatch or TAIZ (ablation) | Interactive image segmentation model (segformerB3) · DAVIS | NoC@85: 3.61 -> 3.48 | Helped |
| Add TAIZ to the baseline without MaskMatch or TAIZ (ablation) | Interactive image segmentation model (segformerB3) · DAVIS | NoC@85: 3.61 -> 3.54 | Helped |
| Add both MaskMatch and TAIZ rather than MaskMatch alone (ablation) | Interactive image segmentation model (segformerB3) · DAVIS | NoC@85: 3.48 -> 3.26 | Helped |
| Replace the MaskMatch loss with the supervised loss (ablation) | Interactive image segmentation model (segformerB3) · Berkeley | NoC@90: 1.72 -> 1.78 | Hurt |
| Set the MaskMatch activation threshold to 0.7 instead of 0.8 (ablation) | MaskMatch · DAVIS | NoC@85: 3.26 -> 3.58 | Hurt |
| Set the MaskMatch activation threshold to 0.9 instead of 0.8 (ablation) | MaskMatch · DAVIS | NoC@85: 3.26 -> 3.49 | Hurt |
| Use previous mask and click points together as the TAIZ guidance map rather than previous mask alone (ablation) | TAIZ · Berkeley | NoC@85: 1.38 -> 1.44 | Hurt |
| Use previous mask and click points together as the TAIZ guidance map rather than click points alone (ablation) | TAIZ · DAVIS | NoC@90: 4.91 -> 4.82 | Helped |
| Use Ours-segformerB3 instead of FocalClick-segformerB3 | Interactive image segmentation · SBD | NoC@85: 3.53 -> 3.55 | Hurt |
| Use Ours-segformerB3 instead of FocalClick-segformerB3 | Interactive image segmentation · DAVIS | SPC: 0.048 -> 0.067 | Hurt |
Surprising findings
- Replacing the matching regularization with the supervised loss yields better NoC@90 on Berkeley (1.78) than the full model's ablation baseline, though it is worse on NoC@85.
Limitations
- Mask matching regularization is only activated when the IoU of the first-step mask exceeds 0.8, so low-quality initial masks provide no regularization signal.
- TAIZ uses the previous mask and clicks as guidance, so it is less effective in early interaction rounds where guidance quality is low (hence the lambda schedule).
Experimental setup
Datasets: COCO LVIS GrabCut Berkeley SBD DAVIS
Benchmarks: GrabCut Berkeley SBD DAVIS
Metrics: NoC@85 NoC@90 NoF@90 IoU@5 BIoU@5 SPC
Baselines: RIS-Net LD CAG BRS FCA f-BRS CDNet RITM EdgeFlow FICI FocalClick
Builds on: FocalClick RITM SegFormer normalized focal loss
Closest papers by meaning
- CFR-ICL: Cascade-Forward Refinement with Iterative Click Loss for Interactive Image Segmentation (AAAI 2024 · similarity 0.87)
- FocSAM: Delving Deeply into Focused Objects in Segmenting Anything (CVPR 2024 · similarity 0.86)
- NTClick: Achieving Precise Interactive Segmentation With Noise-tolerant Clicks (CVPR 2025 · similarity 0.86)
- TETRIS: Towards Exploring the Robustness of Interactive Segmentation (AAAI 2024 · similarity 0.86)
- SAM-REF: Introducing Image-Prompt Synergy during Interaction for Detail Enhancement in the Segment Anything Model (CVPR 2025 · similarity 0.85)
- MaskDiff: Modeling Mask Distribution with Diffusion Probabilistic Model for Few-Shot Instance Segmentation (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.