Research map / Multi-agent systems and games / Human-AI and goal-conditioned
DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?
Yang Chen, Bin Wen, Danyang Peng, Hong-Jie You, Ziqiao Shang, Lan-Zhe Guo
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DualWorldBench, a benchmark for evaluating whether multimodal agents can plan multi-goal deliveries under temporal, physical, and multimodal constraints across symbolic and grounded worlds.
Problem
Existing evaluations of multimodal agents rely on simplified tasks with limited combinatorial structure, so it is unclear whether agents can integrate perception and reasoning for multi-goal planning under temporal, physical, and multimodal constraints.
Approach
A unified benchmark pairing symbolic and grounded versions of delivery planning tasks to test combinatorial multi-goal planning with perception and reasoning integration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use frontier proprietary LLM instead of open-source LLM | LLM planners · TextWorld (4 settings) | average reward: 166.3 -> 385.0 | Improved |
| add online task release (L2) with heterogeneous tasks | open-source LLMs · TextWorld M-L2 | reward vs Offline DP: 23.8% of Offline-DP reward | Hurt |
| add online task release (L2) | proprietary LLMs · TextWorld A-L2 | reward vs Offline DP: 96.1% of Offline-DP reward (278.5 vs. 289.7) | Hurt |
| increase task count to 15 | Llama-3-8B · TextWorld | reward: 0.00 at 15 tasks | Hurt |
| use image-only input instead of text | weaker VLMs (Claude-Haiku-4.5, GPT-5.4-mini) · SimWorld | feasibility: 0.905 -> 0.175 (Haiku) | Hurt |
| use image-only input instead of text | GPT-5.6-Sol · SimWorld | feasibility: 0.968 -> 0.989 | No clear effect |
| use multimodal instead of text input | Qwen3-VL-8B · SimWorld | completion: 0.544 -> 0.974 | Improved |
| increase task count from 5 to 10 | Claude-Opus-5 multimodal · SimWorld | feasibility: 0.730 -> 0.397 | Hurt |
| increase task count | SimWorld models · SimWorld | normalized cost | Hurt |
| evaluate perception/topology vs planning | GPT-5.4-mini · DualWorld-VQA | accuracy: 55.5% / 55.9% -> 6.3% | Hurt |
Experimental setup
Benchmarks: DualWorldBench
Closest papers by meaning
- From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents (ICLR 2026 · similarity 0.90)
- PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks (ICLR 2025 · similarity 0.89)
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.89)
- RoboMP$^2$: A Robotic Multimodal Perception-Planning Framework with Multimodal Large Language Models (ICML 2024 · similarity 0.88)
- Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence (NeurIPS 2025 · similarity 0.88)
- Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
