AI Paper Atlas

Research map / Vision-language and multimodal / Robot action and navigation

DeepPhy: Benchmarking Agentic VLMs on Physical Reasoning

Xinrun Xu, Pi Bu, Ye Wang, Börje F. Karlsson, Ziming Wang, Tengtao Song, Qi Zhu, Jun Song, Zhiming Ding, Bo Zheng

AAAI 2026 · Vision-language and multimodal › Robot action and navigation · approach: Vision-language-action foundation models · 16 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use WM prompt (predict outcome) instead of VLA prompt (ablation)agentic VLMs · PHYRE / I-PHYREsuccess rate: PHYRE Claude 4.0 Opus 7.16 -> 13.33; PHYRE GPT-o3 23.06 -> 21.40; I-PHYRE GPT-o3 81.67 -> 76.67Mixed
use WM prompt instead of VLA prompt (ablation)agentic VLMs · Kinetix (Medium/Large)success rate: Claude 4O Medium w/: 34.72 -> 16.67; Gemini-2.5P Medium w/: 37.50 -> 34.17; GPT-o3 Medium w/o: 36.46 -> 50.00Mixed
add visual annotations of controllable components (ablation)agentic VLMs · Kinetix Small levelsuccess rate: GPT-o3 50.00 -> 63.33; Claude 4O 54.29 -> 68.33Improved
add visual annotations of controllable components (ablation)agentic VLMs · Kinetix Medium/Large levelssuccess rate: Gemini-2.5P Medium 39.58 -> 37.50; GPT-o3 Medium 36.46 -> 40.28No clear effect
allow more attempts (attempt 1 to attempt 10) (ablation)best VLM (GPT-o3) · PHYREsuccess rate (%): 2.87 -> 23.06Improved
use frontier closed-source VLM instead of open-source QwenGPT-o3 vs Qwen-72B · I-PHYRE attempt 10success rate (%): 45.00 -> 81.67Improved
use Qwen open-source VLMs instead of random (MOCK) actionsQwen-72B · I-PHYRE attempt 10success rate (%): 62.50 -> 45.00Hurt
increase Kinetix difficulty from Small to LargeGPT-o3 (VLA, w/o) · Kinetixsuccess rate (%): 50.00 -> 10.00Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy