Research map / Multi-agent systems and games / LLM agents
WorldTravel: A Realistic Multimodal Travel-Planning Benchmark with Tightly Coupled Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces WorldTravel, a 150-task multimodal travel-planning benchmark with a screenshot-based web environment (WorldTravel-Webscape) and tightly coupled temporal constraints, evaluating 10 LLMs and 7 VLMs.
Problem
Existing travel-planning benchmarks use loosely coupled constraints and clean structured data, failing to test whether agents can extract tightly coupled temporal constraints from realistic rendered web interfaces.
Approach
Tool APIs return rendered webpage screenshots instead of structured JSON, and tasks are tuned so tightly coupled timed-entry constraints often leave a unique feasible itinerary, with automatic constraint verification functions.
Key results
- GPT-5.2 achieves 32.67% feasibility rate in the text setting, dropping to 19.33% in the multi-modal screenshot setting.
- With gold hard-constraint parameters provided, GPT-5.2 still reaches only 36.67% feasibility on 30 high-coupling tasks.
- Feasibility collapses from 18.3% at 8-9 constraints to 3.4% at 10-11 constraints.
Surprising findings
- Providing gold constraint parameters in structured form barely improves feasibility (GPT-5.2 unchanged at 36.67%), indicating planning rather than perception is a bottleneck.
- Removing soft constraints yields only modest feasibility gains, and Gemini-3-Pro drops in the multi-modal feasibility-only setting.
Limitations
- Only 150 tasks across five European cities, attraction-centered planning.
- Static synthetic webpages generated with LLMs; deterministic feasibility abstracts away real-world stochasticity.
- Human baseline measured on only 30 tasks with 5 participants.
Experimental setup
Datasets: WorldTravel tasks (150) WorldTravel-Webscape (2,003 webpages)
Benchmarks: WorldTravel
Metrics: Feasibility Rate Constraint Violation Optimality | Feasible
Baselines: TravelPlanner ChinaTravel GPT-5.2 Claude-Opus-4.5 Gemini-3-Pro
Builds on: TravelPlanner ChinaTravel
Closest papers by meaning
- ATLAS: Constraints-Aware Multi-Agent Collaboration for Real-World Travel Planning (ICLR 2026 · similarity 0.91)
- Wide-Horizon Thinking and Simulation-Based Evaluation for Real-World LLM Planning with Multifaceted Constraints (NeurIPS 2025 · similarity 0.89)
- TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios (ICML 2026 · similarity 0.88)
- TravelPlanner: A Benchmark for Real-World Planning with Language Agents (ICML 2024 · similarity 0.87)
- HiMAP-Travel: Hierarchical Multi-Agent Planning for Long-Horizon Constrained Travel (ICML 2026 · similarity 0.87)
- Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.