Research map / Detection and segmentation / Open-set and few-shot detection
Order-aware Interactive Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OIS, an interactive segmentation method that encodes object relative depth as order maps from a depth model and uses order-aware and object-aware masked cross-attention with combined dense and sparse click embeddings, reaching SOTA on HQSeg44K and DAVIS.
Problem
Interactive segmentation methods struggle to separate target objects from background because they lack understanding of relative depth (order) between objects and of object-level foreground/background separation, while dense-fusion methods are slow and sparse-fusion methods misalign prompts with ima
Approach
Builds order maps from a depth map and each click's depth to bias sparse click embeddings toward image regions at similar depth (order-aware attention), adds foreground/background-masked cross-attention for click embeddings (object-aware attention, adapted from Cutie), and combines dense click maps with sparse embeddings for alignment and efficiency.
Key results
- On HQSeg44K (ViT-B 1024), OIS reaches 1-mIoU 89.40 vs 81.79 for SegNext and NoC90 3.95 vs 5.32.
- On DAVIS (ViT-B 1024), OIS reaches NoC95 8.59 vs 10.73 for SegNext and 1-mIoU 87.29 vs 85.97.
- Efficiency: SPC 31 ms vs 58 ms for SegNext, roughly 2x faster per click.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OIS instead of SegNext | Interactive segmentation · HQSeg44K | 1-mIoU: 81.79 -> 89.40 | Helped |
| Use OIS instead of SegNext | Interactive segmentation · DAVIS | NoC95: 10.73 -> 8.59 | Helped |
| Add order-aware attention (ablation) | OIS · DAVIS | 5-mIoU: 91.61 (-1.15) | Helped |
| Add object-aware attention (ablation) | OIS · DAVIS | 5-mIoU: 92.52 (-0.24) | Helped |
| Add sparse fusion (ablation) | OIS · DAVIS | 5-mIoU: 89.81 (-2.95) | Helped |
| Add dense fusion (ablation) | OIS · DAVIS | 5-mIoU: 91.31 (-1.45) | Helped |
| Add order-aware attention (ablation) | OIS · HQSeg44K | 5-mIoU: 92.49 (-1.29) | Helped |
| Add object-aware attention (ablation) | OIS · HQSeg44K | 5-mIoU: 93.28 (-0.5) | Helped |
| Add sparse fusion (ablation) | OIS · HQSeg44K | 5-mIoU: 90.80 (-2.98) | Helped |
| Add dense fusion (ablation) | OIS · HQSeg44K | 5-mIoU: 91.75 (-2.03) | Helped |
| Use object-aware understanding before order-aware understanding (ablation) | OIS · DAVIS | 5-mIoU: 92.49 -> 92.76 | Helped |
| Use both positive- and negative-click order maps rather than positive-only maps (ablation) | OIS · DAVIS | 5-mIoU: 92.04 -> 92.76 | Helped |
Surprising findings
- Object-aware attention removal costs only 0.12 NoC90 on DAVIS, a much smaller effect than order-aware attention removal (+1.04).
- Using a lower-quality depth model (DepthAnything V1, ZoeDepth) yields nearly identical results, suggesting the depth map's precise quality matters little.
Limitations
- Struggles when thin objects occlude the target (failure cases shown).
- Depends on an additional depth model to generate order maps.
- Code release is pending approval per the reproducibility statement.
Experimental setup
Datasets: HQSeg44K COCO LVIS
Benchmarks: HQSeg44K DAVIS
Metrics: NoC90 NoC95 1-mIoU 5-mIoU NoF95 SPC SAT latency
Baselines: RITM FocalClick SimpleClick InterFormer SAM EfficientSAM MobileSAM HQ-SAM HR-SAM SegNext DynaMITe MM-SAM
Builds on: SegNext Cutie SAM Depth Anything V2
Closest papers by meaning
- FocSAM: Delving Deeply into Focused Objects in Segmenting Anything (CVPR 2024 · similarity 0.88)
- NTClick: Achieving Precise Interactive Segmentation With Noise-tolerant Clicks (CVPR 2025 · similarity 0.88)
- AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation (ICLR 2024 · similarity 0.87)
- TETRIS: Towards Exploring the Robustness of Interactive Segmentation (AAAI 2024 · similarity 0.87)
- RClicks: Realistic Click Simulation for Benchmarking Interactive Segmentation (NeurIPS 2024 · similarity 0.86)
- Rethinking Interactive Image Segmentation with Low Latency High Quality and Diverse Prompts (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.