Research map / Multi-agent systems and games / LLM agents

Expanding LLM Agent Boundaries with Strategy-Guided Exploration

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Strategy-Guided Exploration (SGE), which has an LLM agent generate a natural-language strategy before each action, sampled at higher temperature and diversified via positive/negative strategy reflection, to improve RL exploration in agentic tasks.

Problem

Exploration is a central challenge when RL post-trains LLM agents with sparse outcome rewards; the base model's sampling distribution limits discovery of new successful trajectories, so RL mostly refines behaviors the base model already supports.

Approach

Insert a high-level natural-language strategy before each action, sample strategies at a higher temperature than action tokens (mixed-temperature sampling), and condition later rollouts on past successful or failed strategies from the same task (strategy reflection), so exploration happens over diverse outcomes rather than near-duplicate actions.

Key results

• On unseen test tasks, SGE reaches 29.2 (Coding), 36.7 (AndroidWorld), 60.8 (LangR), and 66.6 (AppWorld) pass@1 versus GRPO's 22.0, 21.9, 46.0, and 49.3.
• In Coding, the starting model has pass@2048 of 0.69, GRPO reaches 0.64 final pass@1, and SGE reaches 0.73, exceeding the base model's pass@k ceiling.
• SGE achieves on average 27% higher relative final training success than the next best baseline per environment.

Surprising findings

• Mixed-temperature sampling lowers zero-shot pass@1 but raises pass@16, and this exploration gain translates into better RL training.
• Baselines (GRPO, EntropyAdv, RND, RLAD) do not exceed the base model's maximum pass@k, while SGE does in Coding and LangR.

Limitations

• Requires the base LLM to have sufficient reasoning and planning capacity; the 600M model barely benefits.
• Adds per-step strategy generation cost and latency, limiting real-world deployment.
• Strategy reflection context grows with episode length; evaluated only on environments of roughly tens of steps.

Experimental setup

Datasets: AndroidWorld Language Rearrangement (LangR) Coding hard problems (606 train / 228 test) AppWorld

Benchmarks: AndroidWorld LangR Coding AppWorld

Metrics: pass@1 pass@k

Baselines: GRPO EntropyAdv RND RLAD base model pass@k

Builds on: GRPO DAPO RLAD Random Network Distillation Entropy Advantage

Closest papers by meaning

• LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models (NeurIPS 2025 · similarity 0.91)
• Can large language models explore in-context? (NeurIPS 2024 · similarity 0.91)
• From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems (ICML 2024 · similarity 0.91)
• Learning Agentic Policy from Action Guidance (NeurIPS 2026 · similarity 0.91)
• Representation-Based Exploration for Language Models: From Test-Time to Post-Training (ICLR 2026 · similarity 0.90)
• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration (ICML 2025 · similarity 0.90)