Research map / Vision-language and multimodal / Multimodal LLMs
SCoT: Teaching 3D-LLMs to Think Spatially with Million-scale CoT Annotations
Jinpeng Li, Haiping Wang, Jiabin chen, Yuan Liu, Zhen Dong, Bisheng Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SCoT, a million-scale chain-of-thought annotation dataset for training 3D-LLMs to reason spatially across perception, reasoning, and planning levels.
Problem
3D-LLM training data typically lack explicit reasoning processes, which limits complex spatial reasoning and task planning in 3D environments.
Approach
Annotates a million-scale chain-of-thought dataset spanning three levels (spatial perception, spatial reasoning, task planning) to supply explicit reasoning supervision for 3D-LLMs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CoT supervision to simple perception tasks (ablation) | SCoT-Reasoner · SQA3D / ScanQA | EM / CIDEr: SQA3D EM 55.8 -> 47.4; ScanQA CIDEr 87.9 -> 73.4 | Hurt |
| add CoT supervision to implicit detection (ablation) | SCoT-Reasoner · Implicit Detection | Acc@0.25: 8.6 -> 32.2 | Improved |
| add CoT supervision to implicit detection (ablation) | Chat Scene · Implicit Detection | Acc@0.25: 6.7 -> 26.9 | Improved |
| add CoT supervision to analysis and planning (ablation) | SCoT-Reasoner · Object Analysis | Faithfulness: 5.59 -> 6.15 | Improved |
| add CoT supervision to analysis tasks (ablation) | SCoT-Reasoner · Analysis/planning text similarity | ROUGE-L, METEOR: avg +0.74 ROUGE-L, +0.36 METEOR | No clear effect |
| add CoT supervision to un-situated planning (ablation) | Chat Scene · Un-situated Planning | ROUGE-L: 27.73 -> 27.24 | Hurt |
| keep object-level <SI> tags in CoT (ablation) | SCoT-Reasoner · Object Analysis | Faithfulness: 5.37 -> 6.15 | Improved |
| keep scene-level <SI> tags in CoT (ablation) | SCoT-Reasoner · Situated Planning | Trustworthiness: 6.55 -> 7.14 | Improved |
| train on SCoT instead of 3D-R1 data | Chat Scene · MSQA-ScanNet zero-shot | overall score: 43.1 -> 47.6 | Improved |
| train on SCoT then test zero-shot on ARKitScenes | SCoT-Reasoner · MSQA-ARKitScenes | overall score: 41.0 -> 41.2 | Improved |
| add CoT supervision to scene analysis (ablation) | Chat3D V2 · Scene Analysis | Faithfulness: 6.48 -> 6.43 | No clear effect |
Closest papers by meaning
- Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.90)
- From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.90)
- Spatial Understanding from Videos: Structured Prompts Meet Simulation Data (NeurIPS 2025 · similarity 0.90)
- Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests (NeurIPS 2026 · similarity 0.90)
- Surprise3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenes (NeurIPS 2025 · similarity 0.90)
- 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
