Research map / Vision-language and multimodal / Multimodal LLMs
Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models
Mahtab Bigverdi, Linjie Li, Weikai Huang, Jieyu Zhang, Tuhin Kundu, Zelun Luo, Chris Kim, Jaemin Cho, Ranjay Krishna, Linda Shapiro, Liu Yiming
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| answer-only fine-tuning on AI2-THOR data | Bagel (base) · AI2-THOR PET | accuracy (%): 40.3 -> 97.5 | Improved |
| add IPT supervision vs label-only (ablation) | Bagel · AI2-THOR MVC | accuracy (%): 63.9 -> 67.3 | Improved |
| add IPT supervision vs label-only (ablation) | Bagel · Habitat PET | accuracy (%): 82.0 -> 87.0 | Improved |
| add IPT supervision vs label-only (ablation) | Bagel · AI2-THOR PT (avg) | accuracy (%): 65.7 -> 49.0 | Hurt |
| add IPT supervision vs label-only (ablation) | Bagel · AI2-THOR PET | accuracy (%): 97.5 -> 96.8 | No clear effect |
| add textual chain-of-thought vs label-only (ablation) | Bagel · AI2-THOR PET | accuracy (%): 97.5 -> 83.1 | Hurt |
| add textual chain-of-thought vs label-only (ablation) | Bagel · AI2-THOR PT (avg) | accuracy (%): 65.7 -> 49.7 | Hurt |
| replace Text CoT with IPT (ablation) | Bagel · AI2-THOR PET | accuracy (%): 83.1 -> 96.8 | Improved |
| mix IPT and label-only data vs label-only (ablation) | Bagel · Real PT (different env) | accuracy (%): 54.7 -> 58.6 | Improved |
| mix IPT and label-only data vs IPT (ablation) | Bagel · AI2-THOR MVC | accuracy (%): 67.3 -> 62.3 | Hurt |
| raise imagination resolution Latent-4 to Latent-64 (ablation) | Bagel + IPT · AI2-THOR MVC | accuracy (%): 53.5 -> 63.1 | Improved |
| raise imagination resolution Latent-32 to Latent-64 (ablation) | Bagel + IPT · Habitat PET | accuracy (%): 87.0 -> 83.3 | Hurt |
Closest papers by meaning
- Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators (NeurIPS 2026 · similarity 0.90)
- Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.90)
- Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing (NeurIPS 2025 · similarity 0.89)
- 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.89)
- Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views (CVPR 2026 · similarity 0.89)
- Perception Tokens Enhance Visual Reasoning in Multimodal Language Models (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
