Research map / 3D vision / 3D generation
SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding
Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, Siyuan Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train on SceneVerse before fine-tuning | GPS · ScanRefer Acc@0.5 overall | accuracy: 40.4 -> 47.1 (pre-train) -> 48.1 (fine-tuned) | Improved |
| train GPS from scratch on benchmark data only (ablation) | GPS · Nr3D | overall accuracy: 58.7 vs 3D-VisTA 64.2 | Hurt |
| zero-shot transfer with SceneVerse pre-training versus training from scratch | GPS · SceneVerse-val | overall accuracy: 38.5 -> 59.2 | Improved |
| zero-shot GPS versus 3D-VisTA on existing benchmarks | GPS · Nr3D / Sr3D / ScanRefer | accuracy: Nr3D 35.2 -> 32.4; Sr3D 31.2 -> 33.3 | Mixed |
| add generated template text to pre-training (ablation) | GPS · ScanRefer Acc@0.5 | accuracy: 38.4 -> 46.1 | Improved |
| add LLM-rephrased text on top of templates (ablation) | GPS · ScanRefer Acc@0.5 | accuracy: 46.1 -> 46.3 | No clear effect |
| add human-annotated text on top of generated text (ablation) | GPS · ScanRefer Acc@0.5 | accuracy: 46.3 -> 47.1 | Improved |
| add scene-level alignment loss (ablation) | GPS · SceneVerse-val | overall accuracy: 62.4 -> 66.9 | Improved |
| add masked language modeling loss without scene-level loss (ablation) | GPS · SceneVerse-val | overall accuracy: 65.2 -> 62.4 | Hurt |
| pre-train on synthetic scenes only (Structured3D or ProcTHOR) then test on real scenes (ablation) | GPS · SceneVerse-val | overall accuracy: 64.8 (real) -> 7.0 (S3D) and 4.2 (ProcTHOR) | Hurt |
| pre-train RegionPLC on SceneVerse data | RegionPLC · ScanNet open-vocab segmentation, SPUNet32 | mIoU: 59.6 -> 61.0 | Improved |
| scale pre-training data from 1/8 to full SceneVerse (ablation) | GPS · ScanRefer and SceneVerse-val | accuracy | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- RegionPLC — RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding (CVPR 2024) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations (NeurIPS 2024 · similarity 0.92)
- DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.92)
- Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.91)
- From Thousands to Billions: 3D Visual Language Grounding via Render-Supervised Distillation from 2D VLMs (ICML 2025 · similarity 0.91)
- From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.91)
- G^3-LQ: Marrying Hyperbolic Alignment with Explicit Semantic-Geometric Modeling for 3D Visual Grounding (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
