Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Benchmarking World-Model Learning with Environment-Level Queries
Archana Warrier, Dat Nguyen, Michelangelo Naim, Moksh Jain, Yichao Liang, Karen Schroeder, Cambridge Yang, Joshua B. Tenenbaum, Sebastian Josef Vollmer, Kevin Ellis, Zenna Tavares
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| humans versus reasoning models on the same AutumnBench tasks | Reasoning models (Claude 4 Sonnet, Gemini 2.5 Pro/Flash, o3, Qwen3) · AutumnBench (129 tasks) | score: Humans outperformed all five reasoning models across every environment and task type | Hurt |
| stochastic instead of deterministic environments | Reasoning models · AutumnBench (17 of 43 stochastic) | score: Reasoning models did significantly better in stochastic settings than in deterministic ones | Improved |
| stochastic instead of deterministic environments | Humans · AutumnBench (17 of 43 stochastic) | score: Humans performed nearly identically in both deterministic and stochastic environments | No clear effect |
| spend more compute per problem (higher-cost model) | Reasoning models · AutumnBench masked-frame prediction and planning | score: higher cost improved performance in 37% of masked-frame prediction and planning tasks | Mixed |
| spend more compute per problem (higher-cost model) | Reasoning models · AutumnBench change detection | score: 33% for change detection | Mixed |
| spend more compute per problem across environments | Reasoning models · AutumnBench (43 environments) | score: 25 environments improved monotonically with cost; 18 environments gave no benefit from additional compute | Mixed |
| use reset as an exploration action (human-style) instead of rarely | Humans vs reasoning models · AutumnBench | reset fraction of actions: Humans 12.5% resets and no-ops each; every model under 7% combined, Claude 2.1% | Improved |
| systematic replay of action subsequences around resets (human behaviour) | Humans vs reasoning models · AutumnBench | LCS ratio pre/post reset: human mean 0.637 vs. 0.188--0.360 for models | Improved |
Closest papers by meaning
- Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (ICLR 2025 · similarity 0.90)
- General agents need world models (ICML 2025 · similarity 0.88)
- Evaluating the World Model Implicit in a Generative Model (NeurIPS 2024 · similarity 0.87)
- Test-Time Adaptation for LLM Agents via Environment Interaction (ICLR 2026 · similarity 0.87)
- AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering (NeurIPS 2026 · similarity 0.87)
- WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
