AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

Towards Self-Evolving Agent Benchmarks : Validatable Agent Trajectory via Test-Time Exploration

Dadi Guo, Tianyi Zhou, Dongrui Liu, Chen Qian, Qihan Ren, Shuai Shao, Zhiyuan Fan, Yi R. Fung, Kun Wang, Linfeng Zhang, Jing Shao

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a self-evolving agent benchmark framework that evolves task complexity via test-time exploration and validates agent trajectories to keep benchmarks from hitting ceilings.

Problem

Existing agent benchmarks are quickly saturated by newly developed agents, making it hard to evaluate agent capabilities.

Approach

A trajectory-based, reproducible benchmark complexity evolution method that uses test-time exploration to generate and validate agent trajectories so benchmark tasks can be evolved to harder complexity levels.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
evolve GAIA tasks with TRACE (round 1)GPT-5-Mini · GAIA totalPass@1: 0.455 -> 0.260Hurt
evolve GAIA tasks with TRACE (round 2)Gemini-2.5-Flash · GAIA Level 1Pass@1: 0.471 -> 0.040Hurt
evolve GAIA tasks with TRACE (round 2)DeepSeek-V3.1 · GAIA totalPass@1: 0.418 -> 0.188Hurt
evolve GAIA Level 3 tasks with TRACEKimi-K2, GPT-5-Mini · GAIA Level 3Pass@1: GPT-5-Mini 0.192 -> 0.375; DeepSeek-V3.1 0.231 -> 0.000Mixed
evolve GAIA with TRACE roundsGPT-5-Mini · GAIA avg token lengthtokens: 2864.5 -> 6275.7Improved
evolve AIME-2024 with TRACE (four rounds)Qwen3-235B-A22B · AIME-2024Average accuracy: 0.9400 -> 0.7167Hurt
evolve AIME-2024 with TRACE (round 2)DeepSeek-R1-Distill-Qwen-7B · AIME-2024Average length: 16890.5 -> 14853.4Mixed
add trajectory-agnostic solver as auxiliary validator (ablation)TRACE · GAIA round 2 item countitems retained: 115 -> 69Hurt
evolve GAIA with TRACEGAIA capability mix · Integrative capabilities sharepercent: 4.0 -> 12.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy