Research map / 3D vision / 3D generation
Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness
Haochen Wang, Yucheng Zhao, Tiancai Wang, Haoqiang Fan, Xiangyu Zhang, Zhaoxiang Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add vanilla reconstruction without 3D-awareness (ablation) | Video-3D-LLM · SQA3D | EM: 58.6 -> 58.8 | No clear effect |
| add cross-view reconstruction (ablation) | Video-3D-LLM · ScanRefer | Acc@0.25: 58.1 -> 60.3 | Improved |
| add global-view reconstruction (ablation) | Video-3D-LLM · SQA3D | EM: 58.6 -> 61.6 | Improved |
| add both cross-view and global-view reconstruction with vanilla (ablation) | Video-3D-LLM · ScanQA | CIDEr: 102.1 -> 107.0 | Improved |
| add both pretext tasks to point-based 3D-LLM (ablation) | 3D-LLM · SQA3D | EM: 49.4 -> 55.0 | Improved |
| add global-view reconstruction to point-based 3D-LLM (ablation) | 3D-LLM · SQA3D | EM: 49.4 -> 54.3 | Improved |
| add scene-level 3D point features at input instead of 3D-aware supervision (ablation) | Video-3D-LLM · ScanRefer | Acc@0.25: 58.1 -> 57.8 | No clear effect |
| add L_3D on 50% unlabeled vision-only data (ablation) | Ross3D with 50% text · ScanQA | CIDEr: 101.7 -> 103.2 | Improved |
| raise mask ratio from 0.25 to 0.75 (ablation) | Ross3D · SQA3D | EM: 63.0 -> 61.2 | Hurt |
| apply reconstruction every step (interval 1) instead of every 4 (ablation) | Ross3D · ScanQA | CIDEr: 107.0 -> 104.8 | Hurt |
| filter black blanks from BEV images (ablation) | Ross3D · ScanQA | CIDEr: 104.6 -> 107.0 | Improved |
| raise BEV resolution from 432x432 to 1024x1024 (ablation) | Ross3D · ScanQA | CIDEr: 107.0 -> 106.5 | No clear effect |
Later papers that compared with this method
2 later accepted papers report Ross3D as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.
- Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering (NeurIPS 2026 · better on 6 of 6 shared results)
- OneCanvas: 3D Scene Understanding via Panoramic Reprojection (NeurIPS 2026 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction (CVPR 2026 · similarity 0.90)
- Reconstructive Visual Instruction Tuning (ICLR 2025 · similarity 0.89)
- MLLMs Need 3D-Aware Representation Supervision for Scene Understanding (NeurIPS 2025 · similarity 0.89)
- Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering (NeurIPS 2026 · similarity 0.89)
- Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning (CVPR 2025 · similarity 0.88)
- Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
