Research map / Multi-agent systems and games / LLM agents
Self-evolving language agent learning
Improves language agents via search, step-level value models, reinforcement learning, and self-distillation.
188 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2026) · an approach in LLM agents, keyword cluster “gui · bench · software”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
10 papers in 2023–24 and 174 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 15.4% to 15.0%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · In Praise of Belief Bases: Doing Epistemic Logic Without Possible Worlds (AAAI 2018 · 28 citations)
- 2019 · Multi-Agent Discussion Mechanism for Natural Language Generation (AAAI 2019 · 6 citations)
- 2023 · Reflexion: language agents with verbal reinforcement learning (NeurIPS 2023 · 559 citations)
- 2024 · Building Cooperative Embodied Agents Modularly with Large Language Models (ICLR 2024 · 39 citations)
- 2025 · SQLFixAgent: Towards Semantic-Accurate Text-to-SQL Parsing via Consistency-Enhanced Multi-Agent Collaboration (AAAI 2025 · 14 citations)
- 2026 · Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents (NeurIPS 2026 · 7 citations)
Earliest papers here
- In Praise of Belief Bases: Doing Epistemic Logic Without Possible Worlds (AAAI 2018 · 28 citations)
- On Consensus in Belief Merging (AAAI 2018 · 6 citations)
- Multi-Agent Discussion Mechanism for Natural Language Generation (AAAI 2019 · 6 citations)
What has been tried
37 papers here report 393 measured changes (a component added, removed or swapped, compared with a baseline): helped 71%, no clear effect 5%, hurt 20%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Remove communication | C-WAH · Average Steps | Hurt | Building Cooperative Embodied Agents Modularly with Large Language Models (ICLR 2024 · 39 citations) | |
| Add constrained prompts | Arithmetic (Test) · score | 0.96 -> 1.00 | Helped | Large Language Models Are Neurosymbolic Reasoners (AAAI 2024 · 19 citations) |
| remove rubber duck debugging (COT) from SQLReviewer | BIRD dev · EX | 58.67 -> 58.15 | Hurt | SQLFixAgent: Towards Semantic-Accurate Text-to-SQL Parsing via Consistency-Enhanced Multi-Agent Collaboration (AAAI 2025 · 14 citations) |
| Increase LATS expansion size from n = 5 to n = 10 | HotPotQA · EM | 0.63 -> 0.65 | Helped | Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models (ICML 2024 · 10 citations) |
| add reflection | ProductQA test · total score | 0.771 -> 0.784 | Helped | AGILE: A Novel Reinforcement Learning Framework of LLM Agents (NeurIPS 2024 · 8 citations) |
| Remove agents' profiles | All topologies and tasks · average performance | 3.67% | Hurt | Scaling Large Language Model-based Multi-Agent Collaboration (ICLR 2025 · 7 citations) |
| add step-level Q-value guidance | WebShop, n=5 · average reward | 0.50 -> 0.61 | Helped | Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Models (AAAI 2025 · 5 citations) |
| Increase retrospective-model LoRA rank from 1 to 4 | HotPotQA, N=1 · success rate | 45% -> 48% | Helped | Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization (ICLR 2024 · 5 citations) |
| Remove the programs-of-thought generation phase | TEACh unseen validation · Success | 35.1 -> 29.4 | Hurt | VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought (NeurIPS 2024 · 5 citations) |
| Add oracle locking so an agent retains a correct answer in later rounds | MMLU (Form.Log.) · accuracy | 0.5000 to 0.6825 | Helped | Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? (NeurIPS 2025 · 4 citations) |
| Replace a trained Analyst or Solver with a base agent | PubMedQA · accuracy | 74.20 -> 72.00; 74.20 -> 73.20; 73.40 -> 72.80; 73.40 -> 71.60 | Hurt | SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning (NeurIPS 2025 · 2 citations) |
| Fine-tune Llama 3.1 8B Instruct on the T1 training set | T1 large · Tool Call F1 | 52.11 -> 87.17 | Helped | T1: A Tool-Oriented Conversational Dataset for Multi-Turn Agentic Planning (NeurIPS 2025 · 2 citations) |
What reviewers will expect
Datasets and baselines named in the 174 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ALFWorld · 16 of 174 papers
- HotpotQA · 15 of 174 papers
- WebShop · 10 of 174 papers
- LoCoMo · 9 of 174 papers
- MMLU · 8 of 174 papers
- GPQA · 8 of 174 papers
- MATH · 6 of 174 papers
- LongMemEval · 6 of 174 papers
Baselines
- GPT-4o · 17 of 174 papers
- Reflexion · 11 of 174 papers
- Mem0 · 10 of 174 papers
- A-Mem · 10 of 174 papers
- GRPO · 9 of 174 papers
- RAG · 8 of 174 papers
- ReAct · 8 of 174 papers
- SFT · 7 of 174 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 184 papers; from OpenAlex and the papers' affiliations).
- Shanghai Jiao Tong University · 12 papers
- Tsinghua University · 7 papers
- Google · 7 papers
- Princeton University · 5 papers
- Microsoft · 5 papers
- University of Science and Technology of China · 5 papers
- Massachusetts Institute of Technology · 4 papers
- University of California, Berkeley · 4 papers
- Tencent · 4 papers
- University of Washington · 4 papers
Use it
Download BibTeX of 27 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- LLM multi-agent decision making (194 papers)
- Robust and efficient agent execution (174 papers)
- Agent data synthesis and behavior studies (108 papers)
- Agent safety and attack benchmarks (98 papers)
Elsewhere in the topic
- Agentic benchmarks and orchestration (185 papers)
- Software engineering agent benchmarks (129 papers)
- LLM multi-agent collaboration and safety (84 papers)
- GUI agent memory and trajectories (72 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).