Research map / Multi-agent systems and games / LLM agents
Mixture of Knowledge Minigraph Agents for Literature Review Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes collaborative knowledge minigraph agents (CKMAs) that build small concept-relation graphs from reference abstracts with GPT-3.5-turbo and generate literature review paragraphs via multiple permuted summarization paths selected by ROUGE-1 agreement.
Problem
Automatic literature review (multiple scientific document summarization) requires modeling complex relations between concepts across many reference papers, which LLMs struggle to organize without explicit structure.
Approach
Replace general-purpose knowledge graphs with query-specific knowledge minigraphs extracted iteratively by an LLM under output, scientific-type, and volume constraints, then sample multiple summaries from permuted reference orders (mixture-of-experts style) and route to the one with highest ROUGE-1 agreement with the others.
Key results
- On Multi-Xscience, CKMAs reach 36.41 ROUGE-1 and 8.78 ROUGE-2, beating the best listed baseline SumBlogger (35.40 / 8.40) and GPT-4 (33.21 / 7.61).
- On TAD, CKMAs score 34.16 ROUGE-1 / 6.22 ROUGE-2; on TAS2, 32.31 / 5.36, both best among compared methods.
- Ablation: removing the summarization router drops Multi-Xscience ROUGE-1 from 36.41 to 34.00; MPSA contributes about 4% and KMCA about 2% per the authors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CKMAs instead of SumBlogger | Literature review generation · Multi-Xscience | ROUGE-1: 35.40 -> 36.41 | Helped |
| use CKMAs instead of SumBlogger | Literature review generation · Multi-Xscience | ROUGE-2: 8.40 -> 8.78 | Helped |
| use CKMAs instead of SumBlogger | Literature review generation · TAD | ROUGE-1: 33.90 -> 34.16 | Helped |
| use CKMAs instead of TAG | Literature review generation · TAD | ROUGE-2: 6.16 -> 6.22 | Helped |
| use CKMAs instead of GPT-4 | Literature review generation · TAS2 | ROUGE-1: 30.71 -> 32.31 | Helped |
| use CKMAs instead of PRIMERA | Literature review generation · TAS2 | ROUGE-2: 5.07 -> 5.36 | Helped |
| add KMCA and MPSA (ablation) | CKMAs · Multi-Xscience | ROUGE-1: 32.04 -> 36.41 | Helped |
| add the scientific constraint to KMCA (ablation) | CKMAs · Multi-Xscience | ROUGE-1: 35.69 -> 36.41 | Helped |
| add the volume constraint to KMCA (ablation) | CKMAs · Multi-Xscience | ROUGE-1: 35.50 -> 36.41 | Helped |
| add iterative construction to KMCA (ablation) | CKMAs · Multi-Xscience | ROUGE-1: 35.04 -> 36.41 | Helped |
| add chunk summarization to MPSA (ablation) | CKMAs · Multi-Xscience | ROUGE-1: 34.00 -> 36.41 | Helped |
| add path permutation to MPSA (ablation) | CKMAs · Multi-Xscience | ROUGE-1: 33.29 -> 36.41 | Helped |
Surprising findings
- Ablation shows the iterative minigraph construction gives the largest gain among KMCA components, and the summary router gives the largest gain among MPSA components.
- FineSurE evaluation shows CKMAs improve over GPT-3.5-turbo on conciseness and faithfulness but completeness is moderate for both.
Limitations
- Relies on a single backbone (GPT-3.5-turbo) and closed API, limiting reproducibility.
- Completeness remains moderate for both CKMAs and the backbone, per FineSurE analysis.
- Performance gain is small when the number of references is small, attributed to limited relations.
Experimental setup
Datasets: Multi-Xscience TAD TAS2
Benchmarks: Multi-Xscience TAD TAS2
Metrics: ROUGE-1 ROUGE-2 faithfulness completeness conciseness
Baselines: LexRank TextRank HeterSumGraph GraphSum TAG KGSum Pointer-Generator BertABS SciBertABS HiMAP BART MGSum
Builds on: knowledge graph construction mixture of experts FineSurE DYGIE++
Closest papers by meaning
- CycleResearcher: Improving Automated Research via Automated Review (ICLR 2025 · similarity 0.85)
- KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment (NeurIPS 2025 · similarity 0.85)
- Agent Reviewers: Domain-specific Multimodal Agents with Shared Memory for Paper Review (ICML 2025 · similarity 0.84)
- From Replication to Redesign: Exploring Pairwise Comparisons for LLM-Based Peer Review (NeurIPS 2025 · similarity 0.82)
- Evaluating AI-based Scientific Knowledge Synthesis with Epidemiological Systematic Reviews (NeurIPS 2026 · similarity 0.82)
- Are Key-Phrases All That Reviewers Care About? A Comprehensive Benchmarking of Reviewer Matchmaking Systems (AAAI 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.