Research map / Multi-agent systems and games / LLM agents
From Interactions to Principles: Experience-Driven Self-Distillation for Evolving LLM Agents
Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, Botian Shi
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes experience-driven self-distillation that lets an LLM agent turn its own interaction traces into reusable principles and distill them back into its policy.
Problem
LLM agents are largely stateless across episodes, discarding interaction traces; prior methods either store raw trajectories for case-based reuse or rely on external teacher models for reflections, which limits generalization or leaves the policy unchanged.
Approach
The agent distills reusable principles from its own experience and self-distills them into its policy, avoiding external teacher models and raw trajectory storage (based on the abstract only).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add experience retrieval at inference (ablation) | EvolveR (Qwen2.5-3B) · 7 QA benchmarks | average exact match: 0.340 -> 0.382 | Improved |
| add experience retrieval at inference (ablation) | EvolveR (Qwen2.5-0.5B) · 7 QA benchmarks | average exact match: 0.078 -> 0.150 | Improved |
| add experience retrieval at inference (ablation) | EvolveR (Qwen2.5-1.5B) · 7 QA benchmarks | average exact match: 0.123 -> 0.270 | Improved |
| use own model for distillation instead of GPT-4o-mini (ablation) | EvolveR (Qwen2.5-3B) · 7 QA benchmarks | average exact match: 0.370 -> 0.382 | Improved |
| use own model for distillation instead of GPT-4o-mini (ablation) | EvolveR (Qwen2.5-0.5B) · 7 QA benchmarks | average exact match: 0.220 -> 0.150 | Hurt |
| use own model for distillation instead of GPT-4o-mini (ablation) | EvolveR (Qwen2.5-1.5B) · 7 QA benchmarks | average exact match: 0.290 -> 0.270 | Hurt |
| add RL training to retrieval with distilled principles (ablation) | EvolveR (Qwen2.5-3B) · 7 QA benchmarks | average exact match: 0.357 -> 0.382 | Improved |
| add RL training to a no-experience agent (ablation) | Qwen2.5-3B · 7 QA benchmarks | average exact match: 0.134 -> 0.325 | Improved |
| add principle distillation and retrieval to RL (ablation) | Qwen2.5-3B · 7 QA benchmarks | average exact match: 0.325 -> 0.382 | Improved |
| add principle retrieval to distilled principles with RL (ablation) | Qwen2.5-3B · 7 QA benchmarks | average exact match: 0.340 -> 0.382 | Improved |
| replace Search-R1-instruct with EvolveR | Qwen2.5-3B · Average of 7 QA benchmarks | average exact match: 0.325 -> 0.382 | Improved |
| replace Search-R1-instruct with EvolveR | Qwen2.5-7B · Average of 7 QA benchmarks | average exact match: 0.385 -> 0.417 | Improved |
Closest papers by meaning
- SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents (NeurIPS 2025 · similarity 0.90)
- Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks (NeurIPS 2025 · similarity 0.90)
- Test-Time Adaptation for LLM Agents via Environment Interaction (ICLR 2026 · similarity 0.90)
- Self-evolving LLM agents with in-distribution Optimization (ICML 2026 · similarity 0.89)
- Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.89)
- MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
