Research map / Multi-agent systems and games / LLM agents

Dual-Scale World Models for LLM Agents towards Hard-Exploration Problems

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GLoW, a dual-scale world-model framework for LLM agents that combines a value-ranked trajectory frontier with multi-path advantage reflection to improve exploration on Jericho text games.

Problem

LLM agents struggle with hard-exploration tasks with sparse rewards and huge action spaces, such as Jericho text-based games, where prior RL and MCTS methods need hundreds of thousands of environment interactions.

Approach

Extends Go-Explore with a global world model (a top-k trajectory frontier analyzed by an LLM to estimate achieved and potential state values for selection) and a local world model (Multi-path Advantage Reflection that compares multiple rollouts from the same state to infer semantic advantages at key decision points).

Key results

• GLoW reaches 73.0 on Zork1 versus 51.7 for the best LLM baseline (ICRL) and achieves the best LLM-based score on 7 of 10 Jericho games.
• GLoW obtains 75.0 on Deephome and 73.7 on Ludicorp, nearly matching XTX (77.7 and 78.8) while using 1,000 steps instead of 800,000.
• Ablations show that removing MAR, the LLM-based global analysis, or the trajectory frontier each lowers scores; the n=3 setting performs best.

Surprising findings

• With a valid-action soft constraint, re-implemented LLM baselines reach near-RL scores on Zork1 (ReAct 48.3), in contrast to near-zero scores reported in prior work.
• Removing all GLoW components yields performance similar to standard IGE, so adding multi-path reflection alone gives no clear gain.

Limitations

• Relies on a single closed API LLM (GPT-4.1-mini), so generality across backbones is untested.
• Results have high variance across seeds (e.g., Enchanter ±20.1) with only 3 seeds, and the LLM baselines use only 1,000 steps.
• Compares sample efficiency against RL methods that use far more interactions, under differing protocols.

Experimental setup

Datasets: Jericho benchmark

Benchmarks: Jericho

Metrics: maximum game score (mean ± standard error over 3 seeds)

Baselines: DRRN KG-A2C RC-DQN XTX MC-LAVE MC-DML ReAct Reflexion ICRL IGE

Builds on: Go-Explore Intelligent Go-Explore Reflexion ReAct TRPO

Closest papers by meaning

• UltraHorizon: Benchmarking LLM-Agent Capabilities in Ultra Long-Horizon Scenarios (ICML 2026 · similarity 0.88)
• Expanding LLM Agent Boundaries with Strategy-Guided Exploration (NeurIPS 2026 · similarity 0.87)
• WALL-E: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents (NeurIPS 2025 · similarity 0.87)
• Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models (ICLR 2025 · similarity 0.86)
• AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.86)
• AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.86)