Research map / Vision-language and multimodal / Multimodal LLMs
UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding
Zhenyu Chen, Ronghang Hu, Xinlei Chen, Matthias Nießner, Angel X. Chang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add synthetic 3D pre-training then joint fine-tuning (ablation) | UniT3D · ScanRefer val grounding | Overall Acc@0.5IoU: 36.49 -> 39.14 | Improved |
| add synthetic 3D pre-training then joint fine-tuning (ablation) | UniT3D · ScanRefer val dense captioning | CIDEr F1@0.5IoU: 26.68 -> 30.28 | Improved |
| add seq-to-seq objective jointly with bidirectional objective (from scratch) (ablation) | UniT3D · ScanRefer val grounding | Overall Acc@0.5IoU: 36.49 -> 37.45 | Improved |
| add bidirectional objective jointly with seq-to-seq (from scratch) (ablation) | UniT3D · ScanRefer val dense captioning | CIDEr F1@0.5IoU: 26.68 -> 27.28 | Improved |
| fine-tune jointly with both objectives instead of single objective after pre-training (ablation) | UniT3D · ScanRefer val grounding | Overall Acc@0.5IoU: 38.45 -> 39.14 | Improved |
| fine-tune jointly with both objectives instead of single objective after pre-training (ablation) | UniT3D · ScanRefer val dense captioning | CIDEr F1@0.5IoU: 28.13 -> 30.28 | Improved |
| pre-train on synthetic data cropped to up to 3 objects instead of 1 (ablation) | UniT3D · ScanRefer val grounding | Overall Acc@0.5IoU: 11.02 -> 13.51 | Improved |
| pre-train on synthetic data cropped to up to 5 objects instead of 3 (ablation) | UniT3D · ScanRefer val dense captioning | CIDEr F1@0.5IoU: 5.91 -> 2.81 | Hurt |
| pre-train on up to 5 object crops instead of 3 (ablation) | UniT3D · ScanRefer val grounding | Overall Acc@0.5IoU: 13.51 -> 11.87 | Hurt |
| add synthetic pre-training (ablation) | UniT3D · ScanRefer val grounding | Unique Acc@0.5IoU: 74.78 -> 73.14 | Hurt |
| add synthetic pre-training (ablation) | UniT3D · ScanRefer val grounding | Multiple Acc@0.5IoU: 27.60 -> 31.05 | Improved |
| add synthetic pre-training (ablation) | UniT3D · ScanRefer val | detection mAP@0.5: 53.91 -> 54.03 | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- D3Net — D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding (ECCV 2022) · better on 5 of 5 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Towards CLIP-driven Language-free 3D Visual Grounding via 2D-3D Relational Enhancement and Consistency (CVPR 2024 · similarity 0.88)
- MonoVLM: Monocular 3D Visual Grounding with Vision Language Models (CVPR 2026 · similarity 0.88)
- From Thousands to Billions: 3D Visual Language Grounding via Render-Supervised Distillation from 2D VLMs (ICML 2025 · similarity 0.88)
- DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.88)
- Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.87)
- AugRefer: Advancing 3D Visual Grounding via Cross-Modal Augmentation and Spatial Relation-based Referring (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
