Research map / Multi-agent systems and games / LLM agents

Mitigating Conversational Inertia in Multi-Turn Agents

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Identifies "conversational inertia" (diagonal attention to prior assistant turns) in multi-turn LLM agents and proposes Context Preference Learning (LoRA DPO on long-vs-short context action pairs) plus Clip context management, improving agent scores across eight AgentGym environments and BrowseComp.

Problem

Multi-turn LLM agents degrade as interaction history grows because they imitate their own previous responses as few-shot examples, limiting exploration; existing fixes rely on environment rewards or complex pipelines.

Approach

Actions generated under long vs. short contexts for identical states differ in inertia, so preference pairs (short-context action preferred over long-context action) can be built without environment rewards; at inference, periodically clipping history to the last L rounds reduces inertia and is KV-cache friendly.

Key results

• CPL with Clip context lifts Qwen3-8B 8-environment average from 64.9 (Window) to 72.5, and Clip context gives 68.9 on the base model.
• Clip context improves GPT-4o-mini average from 66.0 (Window) to 71.1 (+5.1).
• CPL reduces diagonal attention by about 11% (and 14.5% for Long context) on Qwen3-8B; general capability (GPQA-Diamond, MMLU-Redux) is preserved.

Surprising findings

• Clip context matches or beats LLM summarization without any extra summarization model calls.
• Preferring 6-round over long-context actions works better than preferring 1-round over 6-round actions.

Limitations

• Clip context can fail in tasks with strong long-range dependencies when critical early information is cleared.
• Optimal clip parameters (L, H) vary across environments and need task-specific tuning.
• Causal link between diagonal attention and performance is not definitively established.

Experimental setup

Datasets: AgentGym environments (eight tasks) BrowseComp

Benchmarks: MAZE ALFWorld WebShop TextCraft Foliage Homegrid 2048 Rope Hanging (RH) BrowseComp GPQA-Diamond MMLU-Redux

Metrics: success rate score diagonal attention ratio proactive rate

Baselines: Window context Long context Summarization context ReSum

Builds on: DPO LoRA ReAct ReSum

Closest papers by meaning

• Can large language models explore in-context? (NeurIPS 2024 · similarity 0.88)
• Expanding LLM Agent Boundaries with Strategy-Guided Exploration (NeurIPS 2026 · similarity 0.86)
• Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning (NeurIPS 2026 · similarity 0.86)
• Large Language Model Agents Are Not Always Faithful Self-Evolvers (ICML 2026 · similarity 0.86)
• Selective Critique for Cost-Aware LLM Agents in Long-Horizon Decision Making (NeurIPS 2026 · similarity 0.86)
• Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training (ICLR 2025 · similarity 0.86)