Research map / Multi-agent systems and games / LLM agents

From History to State: Constant-Context Skill Learning for LLM Agents

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes constant-context skill learning: per-task-family LoRA skill modules trained with SFT and GRPO-style RL on a deterministic tracker's state block and subgoal rewards, so inference uses only the current observation, one-step context, and a compact state block.

Problem

LLM agents on recurring personal workflows repeatedly reprocess long skill prompts and growing histories, costing tokens and privacy-sensitive context; the paper asks whether procedural context can be moved into weights while keeping task progress explicit.

Approach

Learn lightweight LoRA modules per task family from expert trajectories, with inputs formed from the current observation, one-step context, and a deterministically rendered state block; refine with GRPO-style RL using subgoal progress and error rewards aligned to the same tracker fields.

Key results

• With Qwen3-8B, SFT+RL reaches 89.6% unseen success on ALFWorld, 76.8% success on WebShop, and 66.4% unseen success on SciWorld.
• Prompt tokens per turn drop from 1,310 (ReAct full history) to 184 on ALFWorld and from 3,093 to 488 on WebShop for Qwen3-8B.
• WebShop ablation: removing the state block drops success from 76.8% to 61.2%; SFT-only reaches 62.2%.

Surprising findings

• Current observation alone with no training yields near-floor WebShop success (1.2%), while adding a state block raises it to 23.6% without training.

Limitations

• Tracker and reward rules are manually specified per task family and need validation before RL.
• Evaluation limited to text-mode environments; no GUI or multimodal evaluation.
• Per-family modules do not zero-shot transfer to new skills; each new workflow requires a new adapter.

Experimental setup

Datasets: ALFWorld expert trajectories WebShop expert trajectories SciWorld expert trajectories (ETO split)

Benchmarks: ALFWorld WebShop SciWorld

Metrics: success rate score prompt tokens per turn completion tokens per turn total tokens per episode average steps

Baselines: ReAct-1step ReAct-full ETO DMPO GiGPO MemGen Early Experience EMPO2 ACT

Builds on: ReAct LoRA GRPO ETO GiGPO

Closest papers by meaning

• How to Train Your LLM Web Agent: A Statistical Diagnosis (NeurIPS 2025 · similarity 0.90)
• Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.88)
• Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents (ICML 2026 · similarity 0.88)
• Kimi-Dev: Agentless Training as Skill Prior for SWE-agents (ICLR 2026 · similarity 0.88)
• SkillOpt: Executive Strategy for Self-Evolving Agent Skills (NeurIPS 2026 · similarity 0.87)
• Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.87)