Research map / Multi-agent systems and games / LLM agents

ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training

ICML 2026 · Multi-agent systems and games › LLM agents · approach: GUI agent memory and trajectories · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ScaleEnv, which synthesizes executable tool-use environments and verifiable tasks from scratch across 16 domains, and trains Qwen3 models with GRPO to improve zero-shot generalization to tau^2-Bench and VitaBench.

Problem

Interactive environments for training generalist tool-use agents are scarce, and existing synthesis methods lack environmental diversity, reliability, and scalability.

Approach

Automatically builds executable domain foundations (tool and database schemas with code validated by procedural testing), a tool dependency graph, and tasks via dependency-aware graph expansion with LLM-gated domain growth and execution-based verification, using rule-based database-state rewards.

Key results

• Qwen3-SE-32B improves VitaBench Cross from 5.3 to 10.8 and Instore from 22.5 to 34.5 over Qwen3-32B.
• Qwen3-SE-8B improves tau^2-Bench Retail from 38.4 to 50.9 and Airline from 30.5 to 37.5 over Qwen3-8B.
• Domain scaling from N=2 to N=16 domains shows monotonic zero-shot gains, with performance not yet plateaued at 16.

Surprising findings

• Rule-based database-state rewards outperform an LLM-as-a-judge reward (Avg@4 38.5 vs 36.5), which the authors attribute to reward hacking by the judge.

Limitations

• Training relies on synthesized domains that are mostly strongly LLM-generated; evaluation covers only a few benchmark domains.
• Gains on some VitaBench and tau^2 domains are modest (e.g., Qwen3-SE-8B on Cross 3.0 vs 1.5), with small absolute numbers and few runs.
• User simulator is Qwen2.5-72B-Instruct, so results depend on simulator choice.

Experimental setup

Datasets: ScaleEnv synthesized domains and tasks

Benchmarks: tau^2-Bench VitaBench

Metrics: Avg@4 Pass@4 Pass^4

Baselines: Qwen3-8B Qwen3-32B GPT-OSS-120B-A5B Qwen3-235B-A22B-2507 Kimi-K2-0905 Seed-OSS-36B xLAM-2-32B-fc-r LLM-as-a-Judge reward

Builds on: GRPO Qwen3 AutoForge EnvScaler

Closest papers by meaning

• Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.89)
• Scaling Synthetic Task Generation for Agents via Exploration (ICLR 2026 · similarity 0.88)
• GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators (NeurIPS 2026 · similarity 0.88)
• Don't Just Fine-tune the Agent, Tune the Environment (ICLR 2026 · similarity 0.88)
• Democratizing Tool Learning with Environments Fully Simulated by a Free 8B Language Model (NeurIPS 2026 · similarity 0.88)
• Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use (ICML 2026 · similarity 0.88)