Research map / Large language models / Agents, code and math
FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representations
Fedor Rodionov, Abdelrahman Eldesokey, Michael Birsak, John Femiani, Bernard Ghanem, Peter Wonka
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Python code interpreter (ablation) | GPT-4.1 · HSSD Pair Distance | accuracy (%): 56.0 -> 99.0 | Improved |
| add Python code interpreter (ablation) | GPT-4.1 · HSSD View Angle | accuracy (%): 55.0 -> 96.0 | Improved |
| add Python code interpreter (ablation) | GPT-4.1 · HSSD Visibility | accuracy (%): 46.5 -> 86.5 | Improved |
| add Python code interpreter (ablation) | GPT-4.1 · HSSD Free Space | accuracy (%): 16.0 -> 44.0 | Improved |
| add Python code interpreter (ablation) | GPT-4.1 · HSSD Max Box | accuracy (%): 7.5 -> 3.0 | Hurt |
| add Python code interpreter (ablation) | GPT-4.1 · HSSD Shortest Path | accuracy (%): 25.0 -> 12.5 | Hurt |
| replace JSON with equivalent XML layout (ablation) | gpt-oss-120B, gpt-oss-20B, Qwen3-235B · HSSD subset (View Angle, Visibility, Repositioning) | accuracy: +/-3 percentage points | No clear effect |
| permute object labels with identical geometry (ablation) | gpt-oss-120B · Repositioning | accuracy (%): 60.5 -> 40.0 | Hurt |
| add Boxes image alongside JSON (ablation) | GPT-5-mini · Mixed VLM | accuracy (%): 81.38 -> 83.81 | Improved |
| add Icons image alongside JSON (ablation) | GPT-4.1-mini · Mixed VLM | accuracy (%): 59.31 -> 55.88 | Hurt |
| add GenImg image alongside JSON (ablation) | Gemini 3.1 FL · Mixed VLM | accuracy (%): 66.94 -> 63.81 | Hurt |
| remove JSON and use Boxes image only (ablation) | GPT-5-mini · Mixed VLM | accuracy (%): 81.38 -> 31.06 | Hurt |
Closest papers by meaning
- From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs (CVPR 2026 · similarity 0.90)
- Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.90)
- LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models (CVPR 2025 · similarity 0.89)
- SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.89)
- Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Plans (CVPR 2026 · similarity 0.89)
- SpatialLM: Training Large Language Models for Structured Indoor Modeling (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
