Research map / Multi-agent systems and games / LLM agents
Synergistic Multi-Agent Framework with Trajectory Learning for Knowledge-Intensive Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SMART, a four-agent knowledge-intensive framework (intent reconstructor, retriever, fact locator, response generator) trained on Llama-2-7B with LoRA via two-stage Long-Short Trajectory Learning using trajectory head/end tokens.
Problem
Knowledge-intensive tasks need factually consistent LLM responses, but multi-agent retrieval-augmented pipelines suffer from complex query intent, distracting retrieved passages, and poor knowledge utilization, and end-to-end or modular training either accumulates errors or lacks fine-grained agent
Approach
Train a single LLM to play four agents by marking each sub-trajectory with special head/end tokens; first short-trajectory SFT on per-agent data from existing NLP datasets, then long-trajectory SFT on GPT-4-built full trajectories so the model learns both agent behaviour and inter-agent transitions.
Key results
- SMART reaches 73.18 on PubHealth, 65.58 on ARC-C, 42.60 on PopQA, and 41.16 Str_EM on ASQA, beating ChatGPT on PopQA, SQuAD1 and ASQA.
- On SQuAD1.1, INTERACT-7B (31.52) outperforms SMART (27.80).
- SMART with 60k long-trajectory samples outperforms SelfRAG trained on 120k samples on ARC-C, PopQA and ASQA (training data size ablation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the fact Locator during training (ablation) | SMART long-trajectory learning · PopQA | accuracy: 34.31 -> 37.27 | Helped |
| Add the intent reconstructor during training (ablation) | SMART long-trajectory learning · ARC-C | accuracy: 54.94 -> 60.22 | Helped |
| Add the fact Locator at inference (ablation) | SMART (L+S) · PopQA | accuracy: 37.45 -> 42.60 | Helped |
| Add the intent reconstructor at inference (ablation) | SMART (L+S) · ASQA | EM: 35.30 -> 41.16 | Helped |
| Train only the response generator instead of the full agent combination (ablation) | SMART training · PopQA | accuracy: 21.15 vs 37.27 | Hurt |
| Add short-trajectory learning to vanilla LLM (ablation) | Llama-2-7B · PubHealth | accuracy: 9.80 -> 62.00 | Helped |
| Add long-trajectory learning after short-trajectory learning (ablation) | Llama-2-7B · ASQA | EM: 23.95 -> 39.86 | Helped |
| Combine short- and long-trajectory learning (ablation) | Llama-2-7B · PopQA | accuracy: 37.66 -> 42.60 | Helped |
| Increase long-trajectory training data size (ablation) | SMART · ARC-C, PopQA, and ASQA | performance | Helped |
| Use 60k long-trajectory instances rather than 120k (ablation) | SMART · ARC-C, PopQA, and ASQA | performance | Helped |
Surprising findings
- Removing the fact locator or intent reconstructor at inference only mildly hurts PubHealth, which the authors attribute to knowledge from short-trajectory training.
- SMART keeps performance when agents are skipped at inference, whereas end-to-end multi-agent systems reportedly collapse.
Limitations
- Framework executes sequentially with no iterative loop, which may hurt multi-hop retrieval (acknowledged as future work).
- Retriever is frozen and not trained jointly with the agents.
- Knowledge enhancement baselines were re-implemented on Llama-2 by the authors, so comparisons depend on their reproduction.
Experimental setup
Datasets: FEVER QReCC WoW Natural Questions HotpotQA 2WikiMHQA StrategyQA ASQA CosmosQA OpenbookQA Arc-Easy ThoughtSource
Benchmarks: PubHealth ARC-Challenge PopQA SQuAD 1.1 ASQA
Metrics: accuracy Str_EM ROUGE-L Mauve
Baselines: ChatGPT Llama-2-Chat-7B Llama-2-Chat-13B Mistral-Instruct-7B Vicuna-v1.5-13B Alpaca2-7B REPLUG VANILLA INTERACT RAIT Self-RAG MMAgent
Builds on: Self-RAG RAIT INTERACT REPLUG LoRA
Closest papers by meaning
- Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks (NeurIPS 2025 · similarity 0.88)
- Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.87)
- Latent Collaboration in Multi-Agent Systems (ICML 2026 · similarity 0.87)
- AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.87)
- Distilling LLM Agent into Small Models with Retrieval and Code Tools (NeurIPS 2025 · similarity 0.87)
- SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.