Research map / Multi-agent systems and games / LLM agents

Test-Time Adaptation for LLM Agents via Environment Interaction

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes two annotation-free test-time adaptation methods for LLM agents: syntactic alignment (a per-step adaptation vector on final hidden states) and dynamics grounding (persona-driven exploration that yields an in-context world model), evaluated on WebArena, BFCLv3 and Tau-Bench.

Problem

LLM agents generalize poorly to novel environments because of syntactic mismatch (observation format and action syntax) and semantic mismatch (unknown state-transition dynamics), with no annotated trajectories available at deployment.

Approach

Syntactic alignment updates a lightweight bias vector added to the final hidden states with one language-modeling gradient step per turn (reset per episode). Dynamics grounding runs persona-guided exploration before deployment, summarizes state transitions into natural-language rules, filters them with a reasoning model, and places them in the agent's context as an in-context world model.

Key results

• On WebArena multi-site split, GPT-4.1 success rises from 2% to 23% with dynamics grounding; overall average 30% to 35%.
• Dynamics grounding raises BFCLv3 GPT-4.1 from 55.5 to 64.0 and WebArena GPT-4o mini from 12.0 to 18.0, surpassing WMA (13.5).
• Syntactic alignment improves Qwen2.5-14B-Instruct on Tau-Bench airline from 21.6 to 25.2 with a single update step adding about 3% latency.

Surprising findings

• A naive hybrid of syntactic alignment and dynamics grounding underperforms dynamics grounding alone on BFCLv3 (21.0 vs 22.0).
• Dynamics grounding gains are smaller on simple websites like Shopping, where longer context may slightly hurt.

Limitations

• Only evaluated syntactic alignment on the Qwen2.5 model family.
• Dynamics grounding requires explicit state transitions and is not applied to conversational tasks like Tau-Bench.
• Simple combination of the two methods is suboptimal and may interfere.

Experimental setup

Datasets: WebArena BFCLv3 Tau-Bench

Benchmarks: WebArena BFCLv3 Tau-Bench

Metrics: success rate latency gain

Baselines: World-model-augmented (WMA) agent Qwen2.5-14B-Instruct GPT-4.1 GPT-4o mini

Builds on: NNetNav BrowserGym World-model-augmented agent (WMA) test-time training

Closest papers by meaning

• GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators (NeurIPS 2026 · similarity 0.91)
• Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (ICLR 2025 · similarity 0.90)
• Don't Just Fine-tune the Agent, Tune the Environment (ICLR 2026 · similarity 0.90)
• Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.90)
• LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling (NeurIPS 2026 · similarity 0.89)
• AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition (ICML 2026 · similarity 0.89)