Research map / Detection and segmentation / Open-set and few-shot detection
v-CLR: View-Consistent Learning for Open-World Instance Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes v-CLR, a view-consistent learning framework that uses colorized depth and stylized/edge views plus CutLER object proposals to learn appearance-invariant query features for open-world instance segmentation.
Problem
Open-world instance segmentation fails on novel objects with unseen textures because vision networks are biased toward appearance cues learned from known classes.
Approach
Generate appearance-altered views (colorized depth maps, stylized images, edge maps) of each training image, then enforce cosine-similarity consistency between DETR queries across views, matched via class-agnostic object proposals from CutLER, with the EMA-updated natural-image branch as teacher.
Key results
- On VOC→Non-VOC, v-CLR (DINO) reaches 40.9 ARb100 and 34.1 ARm100 vs 31.1/22.0 for DINO-DETR.
- On VOC→UVO, v-CLR (Def-DETR) improves ARb100 by 2.7 points over SWORD and the DINO variant reaches 47.2 ARb100.
- On COCO→LVIS, v-CLR (DINO) achieves 28.4 ARb100 vs 25.2 for DINO-DETR and 23.5 for SWORD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add v-CLR training to DINO-DETR | DINO-DETR · VOC→Non-VOC (COCO val) | ARb@100: 31.1 -> 40.9 | Helped |
| add v-CLR training to DINO-DETR | DINO-DETR · VOC→UVO | ARb@100: 36.5 -> 47.2 | Helped |
| add v-CLR training to DINO-DETR | DINO-DETR · COCO→LVIS | ARb@100: 25.2 -> 28.4 | Helped |
| add v-CLR training to DINO-DETR | DINO-DETR · COCO→Objects365 | ARb@100: 46.4 -> 47.9 | Helped |
| add v-CLR training to DINO-DETR | DINO-DETR · COCO→Objects365 small objects | ARb_s: 28.8 -> 26.2 | Hurt |
| add general object proposals (ablation) | v-CLR · VOC→Non-VOC (COCO val) | ARb@100: +6.6 | Helped |
| add depth and auxiliary transformed views (ablation) | v-CLR with object proposals · VOC→Non-VOC (COCO val) | ARb@100: +2.3 | Helped |
| add cross-view consistency objective (ablation) | v-CLR with transformed views · VOC→Non-VOC (COCO val) | ARb@100: +0.2 | Helped |
| filter paired object queries before computing consistency loss (ablation) | v-CLR · VOC→Non-VOC (COCO val) | ARb@100: +0.7 | Helped |
| add depth maps to natural-image view (ablation) | v-CLR · VOC→Non-VOC (COCO val) | ARb@100: 38.5 -> 40.5 | Helped |
| add stylized images to natural-image view (ablation) | v-CLR · VOC→Non-VOC (COCO val) | ARb@100: 38.5 -> 40.2 | Helped |
| add edge maps as an auxiliary view with natural and depth views (ablation) | v-CLR · VOC→Non-VOC (COCO val) | ARb@100: 40.5 -> 40.5 | No clear effect |
Surprising findings
- Vanilla DINO-DETR with denoising queries is already a surprisingly strong open-world baseline, outperforming prior SWORD on several settings.
- On small objects under COCO→Objects365, v-CLR slightly underperforms vanilla DINO-DETR, attributed to CutLER proposals favoring large objects.
Limitations
- Performance on small objects is unstable; gains on small objects are small or negative for DINO-DETR.
- Relies on off-the-shelf CutLER proposals whose preference for large objects and texture bias may limit the method.
- Benefit of general object proposals diminishes when few unknown objects appear in training images.
Experimental setup
Datasets: COCO 2017 PASCAL VOC LVIS UVO Objects365 CLEVR
Benchmarks: COCO 2017 (VOC→Non-VOC) UVO LVIS Objects365 CLEVR
Metrics: AR@10 AR@100 box AR mask AR AR small/medium/large
Baselines: Mask-RCNN CutLER OLN LDET GGN UDOS GOOD SWORD Deformable-DETR DINO-DETR
Builds on: DINO-DETR Deformable-DETR CutLER SWORD
Closest papers by meaning
- OW-VISCapTor: Abstractors for Open-World Video Instance Segmentation and Captioning (NeurIPS 2024 · similarity 0.87)
- Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance (CVPR 2024 · similarity 0.87)
- Solving Instance Detection from an Open-World Perspective (CVPR 2025 · similarity 0.86)
- OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Views (ICLR 2024 · similarity 0.86)
- The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models (ICLR 2024 · similarity 0.86)
- VL-SAM-V2: Open-World Object Detection with General and Specific Query Fusion (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.