Research map / Multi-agent systems and games / LLM agents
diff History for Neural Language Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies the Unix diff operator to consecutive text observations in interaction histories so LM agents see salient changes, improving efficiency in BabyAI-Text and reaching SOTA neural NetHack performance with 1800x less data.
Problem
LM-based embodied agents face long, verbose textual prompts when observations are converted to text and combined with interaction history, limiting domains, history length, and low-sample instruction finetuning.
Approach
Replace full-text observations after the first step in a history with Unix diff deltas between consecutive observations, a task-agnostic relational abstraction that also compresses tokens in high-dimensional observation settings.
Key results
- On NetHack, a GPT-2 127M diff-history agent reaches mean score 1885 ± 213 with 32M expert actions, comparable to the 115B-keypress neural SOTA score of 2740.
- Diff history reduces NetHack token count per observation by 460% and needs at least 25% fewer FLOPs than full-text history on BabyAI-Text low-sample finetuning.
- Ablating diff observations drops NetHack mean score by 98% (from ~1885 to ~31-ish range for full-text history agent, 31 vs 1885 in table).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| diff history observations instead of full-text observations (ablation) | GPT-2-127M language agent · NetHack | mean score: 1885 vs 31 mean score | Helped |
| add auxiliary causal world-model objective to diff-history agent (ablation) | diff history GPT-2 agent · NetHack | mean score: 1094 vs 1203/1885 without world model | Hurt |
| add auxiliary world-model objective to full-text history agent (ablation) | full-text history GPT-2 agent · NetHack | mean score: 344 vs 31 | Helped |
| large-scale pretraining removed (train from scratch) (ablation) | diff history GPT-2 agent · NetHack | mean score: 36% reduction in mean score | Hurt |
| double the number of expert-action demonstrations (1M chunks) | diff history GPT-2 agent · NetHack | mean score: 2766 vs 1885 (47% higher) | Helped |
| diff history instead of full-text history (ablation) | GPT-2-127M in BabyAI-Text · BabyAI-Text | gradient steps / FLOPs to reach peak reward: >=25% fewer FLOPs | Helped |
| diff history increases per-observation token count in BabyAI-Text | diff history representation · BabyAI-Text | tokens per observation: 210% average-case increase | Hurt |
| diff history compresses observation tokens in NetHack | diff history representation · NetHack | tokens per observation: 460% reduction in average token count per timestep | Helped |
Surprising findings
- Diff history increases per-observation tokens by 210% in BabyAI-Text, yet still improves finetuning efficiency.
- Adding an auxiliary world-model objective helps full-text agents by roughly 10x but hurts diff-history agents.
Limitations
- Assumes observations and actions can be fully expressed as text; scaling to other modalities is open.
- Relies on structure present in textual observations; unlikely to help unstructured narrative text.
- Best agents do not use explicit planning.
Experimental setup
Datasets: BabyAI-Text NetHack Learning Environment bot demonstrations (LangHack)
Benchmarks: BabyAI-Text NetHack Learning Environment (NetHack Challenge task)
Metrics: mean rollout reward NetHack game score (mean and median) FLOPs/optimizer steps to peak performance
Baselines: full-text history agents vision agents (LSTM CDGPT) Transformer vision agent NetHack SOTA neural agent (115B keypresses)
Builds on: Unix diff GPT-2 NetHack Learning Environment BabyAI-Text NetHack Language Wrapper
Closest papers by meaning
- EvoLM: In Search of Lost Language Model Training Dynamics (NeurIPS 2025 · similarity 0.86)
- Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks (ICLR 2024 · similarity 0.85)
- Recursive Introspection: Teaching Language Model Agents How to Self-Improve (NeurIPS 2024 · similarity 0.85)
- AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.85)
- Latent Action Reparameterization for Efficient Agent Inference (NeurIPS 2026 · similarity 0.85)
- Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.