Research map / Multi-agent systems and games / LLM agents

GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GenEnv, a co-evolutionary game where a generative LLM environment simulator is rewarded (via an alpha-curriculum reward) for producing tasks whose success rate stays near 0.5 while the agent trains with GRPO.

Problem

Training LLM agents requires costly real-world interaction data, and static synthetic or expert datasets fail to adapt to the agent's evolving weaknesses.

Approach

A two-player curriculum game in which an LLM environment simulator generates tasks on-the-fly and is updated via reward-weighted regression with a bell-shaped reward peaking at a target agent success rate (alpha=0.5), while the agent is updated with GRPO on valid simulated traces.

Key results

• GenEnv (7B) averages 53.6 across five benchmarks versus 33.0 for Qwen2.5-7B and 48.8 for Qwen 2.5 72B.
• On ALFWorld GenEnv reaches 54.5 vs 14.2 for the Qwen2.5-7B base model (+40.3 points).
• GenEnv reaches validation score 0.458 on BFCL with 1x data, above Gemini-Offline 3.3x at 0.438.

Surprising findings

• Adding 3.3x more Gemini 2.5 Pro synthetic data underperforms GenEnv trained with a small self-generated simulator curriculum.
• The 7B GenEnv agent's average exceeds several 14B-72B open models in the reported table.

Limitations

• The theoretical analysis relies on stylized bandit assumptions (bounded score variation) and does not fully characterize LLM training dynamics.
• Gains are reported against baselines trained under differing settings (e.g., GenEnv-Static uses PPO), limiting controlled comparison.
• Compute and training cost of the co-evolving simulator are not reported.

Experimental setup

Datasets: API-Bank ALFWorld BFCL Bamboogle TravelPlanner

Benchmarks: ALFWorld BFCL API-Bank Bamboogle TravelPlanner

Metrics: accuracy / success rate TravelPlanner CS and HD micro/macro average validation score

Baselines: Qwen2.5-7B-Instruct ReSearch SearchR1 ToRL Qwen3-14B GPT-OSS-20B Gemini-Offline augmentation GenEnv-Random GenEnv-Static

Builds on: GRPO reward-weighted regression zone of proximal development ToRL

Closest papers by meaning

• Test-Time Adaptation for LLM Agents via Environment Interaction (ICLR 2026 · similarity 0.91)
• Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning (ICML 2026 · similarity 0.89)
• Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.89)
• Don't Just Fine-tune the Agent, Tune the Environment (ICLR 2026 · similarity 0.88)
• ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training (ICML 2026 · similarity 0.88)
• Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.88)