Research map / Multi-agent systems and games / LLM agents

WebGym: Scaling Training Environments for Long-Horizon Visual Web Agents with Realistic Tasks

CVPR 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces WebGym, a ~300k-task open-source training environment with rubric-based LLM evaluation and an asynchronous rollout system, and trains Qwen3-VL-8B-Instruct with a filtered-BC REINFORCE variant to 42.9% on an unseen-website OOD test set.

Problem

Visual web agents lack large, diverse, realistic training task sets with reliable evaluators, and rollout collection on live browsers is too slow to scale online RL.

Approach

Build a large task set by generating fact-group rubrics with GPT-4o and decomposing tasks into easier subsets of groups, pair it with an asynchronous server/client browser rollout system, and train with REINFORCE using only successful trajectories plus memory prompting and a repeated-screenshot penalty.

Key results

• Qwen3-VL-8B-Instruct success on OOD test set rises from 26.2% to 42.9% after RL on WebGym, versus 27.1% for GPT-4o and 29.8% for GPT-5-Thinking.
• Asynchronous rollout system achieves 4-5x rollout speedup; 64 CPUs collect 1.8k rollouts in 48.6 minutes vs 264 minutes for synchronous system.
• Shortening training horizon from (15,30,45) to (10,20,30) raises peak success from 38.2% to 42.9%.

Surprising findings

• Training only on easy tasks outperformed training only on medium tasks and was more stable than upweighting harder tasks.
• The Thinking variant of Qwen3-VL-8B started higher but the Instruct model surpassed it as RL progressed.

Limitations

• LLM-generated rubrics can be overly strict, reducing recall and sample efficiency.
• Thinking variant is costlier to train and was not effectively exploited by the RL recipe.
• Proprietary model comparison on GPT-5 used only a 300-task subset due to cost.

Experimental setup

Datasets: InSTA-v3 PAE-WebVoyager AgentSynth BrowseComp GAIA-Web TravelPlanner DeepShop Mind2Web-Live Online Mind2Web Mind2Web-2

Benchmarks: WebGym OOD test set (1,167 tasks)

Metrics: success rate

Baselines: GPT-4o GPT-5-Thinking Qwen3-VL-Thinking-8B Qwen3-VL-Instruct-8B (zero-shot)

Builds on: REINFORCE WebRL Proposer-Agent-Evaluator (PAE) AgentSynth Online Mind2Web GLM-4.1V

Closest papers by meaning

• WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents (ICLR 2026 · similarity 0.90)
• How to Train Your LLM Web Agent: A Statistical Diagnosis (NeurIPS 2025 · similarity 0.90)
• WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning (ICLR 2025 · similarity 0.90)
• WebWorld: A Large-Scale World Model for Web Agent Training (ICML 2026 · similarity 0.90)
• Multimodal Web Navigation with Instruction-Finetuned Foundation Models (ICLR 2024 · similarity 0.90)
• A Subgoal-driven RL Framework for Improving Long-Horizon Web Agents (NeurIPS 2026 · similarity 0.89)