Research map / Multi-agent systems and games / LLM agents

Real-Time Reasoning Agents in Evolving Environments

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Real-Time Reasoning Gym (three real-time games with token-based time pressure) and AgileThinker, which runs a planning thread and a reactive thread in parallel so the reactive thread can read partial planning traces, outperforming reactive-only and planning-only DeepSeek agents.

Problem

LLM agents are typically evaluated in environments that pause while the model reasons, so they cannot make timely decisions when the world keeps evolving during computation.

Approach

Environment steps at a fixed token-based rate regardless of agent reasoning; a planning LLM streams long reasoning over a frozen state while a reactive LLM, under a strict token budget, emits actions using the latest observation plus the partial planning trace.

Key results

• Under high cognitive load at 8k tokens/step, AgileThinker scores 0.50 versus 0.15 for the reactive agent on average (per the reported averages: 0.88 to 0.50 vs 0.89 to 0.15 across load).
• Under wall-clock experiments with DeepSeek API (T_E = 6 min), AgileThinker achieves 0.88 on Freeway, 0.45 on Snake, and 0.89 on Overcooked, above reactive V3 and planning R1.
• Token count correlates linearly with wall-clock time (R^2 = 0.9986; alpha = 0.0473 s/token).

Surprising findings

• R1 with budget forcing performs worse than the non-thinking V3 reactive agent (0.01 vs 0.39), since forced truncation typically yields no-op actions.
• Budget-aware generation methods only minorly shift token usage and still cause severe performance drops under truncation.

Limitations

• Experiments only on DeepSeek models because other providers do not expose reasoning traces and open models perform poorly.
• No evidence that the dual system models human dual process cognition.
• Games are simplified simulations of real-world dynamics.

Experimental setup

Datasets: Freeway Snake Overcooked

Benchmarks: Real-Time Reasoning Gym (Freeway, Snake, Overcooked)

Metrics: normalized game score paired t-test p-value

Baselines: Reactive agent (DeepSeek-V3) Planning agent (DeepSeek-R1) Code-as-Policy (R1) Reactive (R1) with budget forcing DeepSeek-V3.2 reactive/planning

Builds on: budget forcing code-as-policy OpenAI Gym dual process theory

Closest papers by meaning

• Tools as Continuous Flow for Evolving Agentic Reasoning (NeurIPS 2026 · similarity 0.90)
• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory (ICLR 2026 · similarity 0.89)
• Benefits and Limitations of Communication in Multi-Agent Reasoning (ICLR 2026 · similarity 0.88)
• A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.88)
• Agentick: A Unified Benchmark for General Sequential Decision-Making Agents (NeurIPS 2026 · similarity 0.87)
• The Era of Agentic Organization: Learning to Organize with Language Models (NeurIPS 2026 · similarity 0.87)