Research map / 3D vision / 3D generation
Unifying 3D Vision-Language Understanding via Promptable Queries
ziyu zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, Qing Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add point features to voxel features (ablation) | PQ3D · ScanRefer / SQA3D / Scan2Cap | acc / acc / CIDEr: +3.1%, +1.7%, +6.8% | Improved |
| add image features to voxel and point features (ablation) | PQ3D · ScanRefer / SQA3D / Scan2Cap | acc / acc / CIDEr: +2.0%, +1.7%, +5.7% | Improved |
| joint multi-task training instead of single-dataset training (ablation) | PQ3D · Scan2Cap | CIDEr@0.5: 75.6 -> 80.3 | Improved |
| joint multi-task training instead of single-dataset training (ablation) | PQ3D · ScanRefer | acc@0.5: 47.4 -> 51.2 | Improved |
| joint training on Multi3DRefer only vs unified training (ablation) | PQ3D · Multi3DRefer | ZT F1: ZT 61.1 vs 57.7; avg 48.6 vs 50.1 | Mixed |
| use a 4-layer query decoder rather than 2 or 6 layers (ablation) | PQ3D · ScanRefer | acc@0.5: 50.2 (2-layer), 50.7 (6-layer) -> 51.2 | Improved |
| use the proposed decoder instead of sequential cross-attention (ablation) | PQ3D · Nr3D | accuracy (overall): 56.1 -> 66.7 | Improved |
| add Refer task data to joint training (ablation) | PQ3D · QA / Caption | gain: +1.8% QA, +2.2% caption | Improved |
| add QA task data to joint training (ablation) | PQ3D · Refer | accuracy: +0.0 | No clear effect |
| train with random feature dropout, infer without image features (ablation) | PQ3D · ScanRefer / SQA3D / Scan2Cap | acc / acc / CIDEr: ScanRefer 49.2 -> 49.4 | No clear effect |
| use closed-vocabulary classification head instead of prompted CLIP text | PQ3D · ScanNet200 val | AP: 20.2 -> 27.0 | Improved |
| 3D PQ3D features for navigation agent versus baseline VC-1 | VC-1 (ViT-B) · ObjNav CortexBench | success rate: 57.1 -> 80.0 | Improved |
Closest papers by meaning
- Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysis (CVPR 2025 · similarity 0.90)
- GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models (ICLR 2026 · similarity 0.90)
- Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors (NeurIPS 2025 · similarity 0.90)
- VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction (CVPR 2026 · similarity 0.90)
- Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning (CVPR 2026 · similarity 0.90)
- Proxy3D: Efficient 3D Representations for Vision-Language Models via Semantic Clustering and Alignment (CVPR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
