Research map / Multi-agent systems and games / LLM agents

TimeWarp: Evaluating Web Agents by Revisiting the Past

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces TimeWarp, a containerized benchmark of three websites each with six UI/code versions from different web eras, and TimeTraj, a human-in-the-loop plan distillation method that yields teacher trajectories across versions, plus TimeWarp-BC training on full teacher responses (action, thinking, planning, memory tokens).

Problem

Web agents are evaluated on static environments, so it is unknown whether agents trained and tested today remain effective when website UI and frontend code change over time; collecting human trajectories for every web version is costly.

Approach

Build six historically-styled frontend versions per website with fixed tasks so performance shifts are attributable to version changes; collect multi-version trajectories by having a teacher agent execute human-refined version-independent plans, and train agents by behavior cloning on the full teacher response (action, thinking, planning, memory) rather than actions only.

Key results

• Qwen-3 4B improves from 20.4% zero-shot AXT to 37.7% with TimeWarp-BC.
• Llama-3.1 8B improves from 0.7% zero-shot AXT to 27.0% with TimeWarp-BC.
• Visual SoM zero-shot performance for Qwen-3 VL 8B spans 2.6% to 21.4% across versions, versus 7.4 to 8.1 point spread for HTML/AXT.

Surprising findings

• Vanilla behavior cloning on a single version degrades Qwen-3 4B performance relative to zero-shot in some settings, while adding non-action tokens via TimeWarp-BC helps.
• Qwen-3 VL 8B Thinking underperforms its non-thinking counterpart across all training settings.

Limitations

• null

Experimental setup

Datasets: TimeWarp training split WebArena-Lite

Benchmarks: TimeWarp WebArena-Lite

Metrics: success rate

Baselines: HTML zero-shot AXT zero-shot BC single version (BCv6) BC multi-version (BCv1:6) BC with thinking tokens (BCT) Qwen-3VL 8B Gemma-3 12B Llama-3.1 8B

Builds on: BrowserGym WebArena WebShop behavior cloning

Closest papers by meaning

• REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites (NeurIPS 2025 · similarity 0.90)
• WARC-Bench: Web Archive based Benchmark for GUI Subtask Executions (ICLR 2026 · similarity 0.88)
• WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Exploration (AAAI 2025 · similarity 0.88)
• RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Users (AAAI 2026 · similarity 0.87)
• ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents (ICLR 2026 · similarity 0.87)
• AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines (ICML 2026 · similarity 0.87)