Research map / Detection and segmentation / 3D detection and driving
OV-Uni3DETR: Towards Unified Open-Vocabulary 3D Object Detection via Cycle-Modality Propagation
Zhenyu Wang, Ya-Li Li, TAICHI LIU, Hengshuang Zhao, Shengjin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 2D to 3D semantic propagation from a 2D open-vocab detector (ablation) | OV-Uni3DETR · SUN RGB-D (P+I) | AP25 novel: N/A -> 11.21 | Improved |
| add 3D to 2D geometric propagation on top of 2D to 3D (ablation) | OV-Uni3DETR · SUN RGB-D (image-only) | AP25 novel: 3.86 -> 5.41 | Improved |
| add 3D to 2D geometric propagation on top of 2D to 3D (ablation) | OV-Uni3DETR · SUN RGB-D (P+I) | AP25 novel: 11.21 -> 12.96 | Improved |
| add switched-modality training (ablation) | OV-Uni3DETR · SUN RGB-D (image-only) | AP25 novel: 1.12 -> 5.41 | Improved |
| add 3D convolution with BN to regularize modality features (ablation) | OV-Uni3DETR · SUN RGB-D (image-only) | AP25 novel: 3.37 -> 5.41 | Improved |
| add dual-branch output layers for noisy 3D boxes (ablation) | OV-Uni3DETR · SUN RGB-D (point cloud) | AP25 novel: 6.34 -> 9.66 | Improved |
| multi-modal switched training vs single-modality training, point cloud inference (ablation) | OV-Uni3DETR · SUN RGB-D (point cloud) | AP25 novel: 9.64 -> 9.66 | No clear effect |
| unified multi-modal detector vs CoDA, point cloud inference | OV-Uni3DETR · SUN RGB-D | AP25 novel: 6.71 -> 9.66 | Improved |
| RGB-only inference vs CoDA with point clouds | OV-Uni3DETR · ScanNet (60 classes) | AP25 novel: 6.54 -> 8.10 | Improved |
| OV-Uni3DETR vs OV-3DET and CoDA with no seen classes | OV-Uni3DETR · ScanNet 20-class | mean AP25: 19.32 -> 25.33 | Improved |
| image-only RGB training/inference vs point cloud input for 2D boxes | OV-Uni3DETR · SUN RGB-D 2D detection | AP novel (2D): 5.21 (I) vs 0.40 (P) | Hurt |
Later papers that compared with this method
4 later accepted papers report OV-Uni3DETR as a baseline in their results tables; 4 of them report a better number on most of the results they share with it.
- Open-Vocabulary 3D Object Detection with Co-Distillation Discovery and Dual Guidance Robust Training (ECCV 2026 · better on 7 of 7 shared results)
- BEVOpen3D: Towards Open-World 3D Object Detection in Bird's-Eye-View (ECCV 2026 · better on 3 of 4 shared results)
- Zoo3D: Zero-Shot 3D Object Detection at Scene Level (CVPR 2026 · better on 3 of 3 shared results)
- Towards 3D Objectness Learning in an Open World (NeurIPS 2025 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ImOV3D: Learning Open Vocabulary Point Clouds 3D Object Detection from Only 2D Images (NeurIPS 2024 · similarity 0.93)
- FM-OV3D: Foundation Model-Based Cross-Modal Knowledge Blending for Open-Vocabulary 3D Detection (AAAI 2024 · similarity 0.92)
- Towards 3D Objectness Learning in an Open World (NeurIPS 2025 · similarity 0.92)
- Training an Open-Vocabulary Monocular 3D Detection Model without 3D Data (NeurIPS 2024 · similarity 0.91)
- Hierarchical Cross-Modal Alignment for Open-Vocabulary 3D Object Detection (AAAI 2025 · similarity 0.91)
- Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
