Research map / Multi-agent systems and games / LLM agents
CubeBench: Diagnosing Interactive, Long-Horizon Physical Intelligence under Partial Observations
Huan-ang Gao, Zikang Zhang, Tianwei Luo, Kaisen Yang, Xinzhe Juan, Jiahao Qiu, Tianxing Chen, Bingxiang He, Hao Zhao, Hao Zhou, Shilong Liu, Mengdi Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CubeBench, a benchmark for diagnosing LLM agents' physical intelligence on long-horizon, partially observed interactive tasks, targeting spatial reasoning, state tracking via mental simulation, and active exploration.
Problem
LLM agents struggle to transfer from digital tasks to the physical world because they must form and maintain a robust spatial mental model under partial observation over long horizons.
Approach
Isolates three cognitive challenges (spatial reasoning, long-horizon state tracking via mental simulation, active exploration under partial observation) in a diagnostic benchmark so each can be evaluated separately.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dense face reward to Basic Agent (ablation) | GPT-5 · CubeBench Full Symbolic short-horizon | pass rate: 0.75 -> 0.85 | Improved |
| add dense sticker or heuristic reward to Basic Agent (ablation) | GPT-5 · CubeBench Full Symbolic short-horizon | pass rate: 0.75 -> 0.65 (sticker), 0.50 (heuristic) | Hurt |
| add dense rewards (any type) on long-horizon tasks (ablation) | all models · CubeBench long-horizon | pass rate: 0.00 -> 0.00 | No clear effect |
| add dense sticker reward (ablation) | Claude Sonnet 4 · CubeBench Full Symbolic short-horizon | pass rate: 0.05 -> 0.25 | Improved |
| add dense face reward to Basic Agent (ablation) | GPT-5 · CubeBench Full Visual short-horizon | pass rate: 0.20 -> 0.55 | Improved |
| add Standard-Solver tool (ablation) | GPT-5 · CubeBench Full Symbolic long-horizon | pass rate: 0.00 -> 0.95 | Improved |
| add Standard-Solver tool (ablation) | Claude Sonnet 4 · CubeBench Full Symbolic long-horizon | pass rate: 0.00 -> 0.85 | Improved |
| replace Standard-Solver with Ideal-Solver (auto format conversion) (ablation) | Gemini 2.5 Pro · CubeBench Full Symbolic short-horizon | pass rate: 0.70 -> 1.00 | Improved |
| replace Standard-Solver with Ideal-Solver (ablation) | Claude Sonnet 4 · CubeBench Full Visual | pass rate: 0.00 -> 0.00 | No clear effect |
| add solver tools on Vertex view (ablation) | all models · CubeBench Vertex view | pass rate: 0.00 -> 0.00 | No clear effect |
| remove code execution (No-Code mode) (ablation) | GPT-5 · CubeBench Full Symbolic short-horizon | pass rate: 0.75 -> 0.25 | Hurt |
| remove code execution (No-Code mode) (ablation) | Grok-4 · CubeBench Full Symbolic short-horizon | pass rate: 0.20 -> 0.30 | Improved |
Experimental setup
Benchmarks: CubeBench
Closest papers by meaning
- UltraHorizon: Benchmarking LLM-Agent Capabilities in Ultra Long-Horizon Scenarios (ICML 2026 · similarity 0.88)
- EvoEmpirBench: Dynamic Spatial Reasoning with Agent-ExpVer (AAAI 2026 · similarity 0.88)
- Reasoning in Visual Navigation of End-to-end Trained Agents: A Dynamical Systems Approach (CVPR 2025 · similarity 0.88)
- UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks (NeurIPS 2026 · similarity 0.87)
- From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs (CVPR 2026 · similarity 0.87)
- LTD-Bench: Evaluating Large Language Models by Letting Them Draw (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
