Research map / Multi-agent systems and games / LLM agents
SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SiriuS, a self-improving framework that builds an experience library of successful multi-agent reasoning trajectories (plus augmented failed ones) and fine-tunes each agent via SFT on GPT-3.5/GPT-4o-mini.
Problem
Optimizing LLM-based multi-agent systems is hard because manually designed prompts are fragile and training data for specialized agents is scarce, with unclear credit assignment across agents.
Approach
Retain interaction trajectories that yield correct final outcomes as per-agent SFT data (an experience library), and augment failed trajectories by having an external agent give ground-truth-informed feedback, regenerating, and rephrasing the solution as direct reasoning; iterate fine-tuning across agents in problem-solving, actor-critic, and competitive settings.
Key results
- On PubMedQA, SiriuS with GPT-3.5-turbo reaches 74.20% vs 56.40% single-agent and 71.80% for CoMM.
- On College Chemistry with GPT-3.5-turbo, SiriuS reaches 56.92% vs 38.46% single-agent.
- In the Actor-Critic PubMedQA setting with GPT-4o-mini, SiriuS reaches 66.80% overall accuracy vs 58.20% for Prompt.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SiriuS instead of the strongest listed baseline | GPT-3.5-turbo and GPT-4o-mini problem-solving systems · College Physics, College Chemistry, and PubMedQA | accuracy: 32.71 -> 33.64; 50.77 -> 56.92; 71.80 -> 74.20; 42.99 -> 46.73; 49.23 -> 60.00; 70.60 -> 73.40 | Helped |
| Use SiriuS instead of Prompt | Actor-Critic system · PubMedQA | TP Accuracy: 18.40 -> 35.00; 51.60 -> 59.80 | Helped |
| Use SiriuS instead of Prompt | Actor-Critic system · PubMedQA | Overall Accuracy: 47.60 -> 50.60; 58.20 -> 66.80 | Helped |
| Replace a trained Analyst or Solver with a base agent (ablation) | SiriuS problem-solving system · PubMedQA | accuracy: 74.20 -> 72.00; 74.20 -> 73.20; 73.40 -> 72.80; 73.40 -> 71.60 | Hurt |
| Use one shared fine-tuned LLM for multiple roles instead of separate role-specific agents (ablation) | SiriuS problem-solving system · PubMedQA | accuracy: 74.20 -> 70.40; 73.40 -> 72.00 | Hurt |
| Remove trajectory augmentation (ablation) | SiriuS problem-solving system · PubMedQA | accuracy: 74.20 -> 73.40; 73.40 -> 72.20 | Hurt |
| Add a fine-tuning iteration (ablation) | SiriuS problem-solving system · PubMedQA | accuracy: 74.20 -> 75.00; 73.40 -> 73.60 | Helped |
| Replace trained Actor with a base Actor (ablation) | SiriuS Actor-Critic system · PubMedQA | Overall Accuracy: 50.60 -> 49.00; 66.80 -> 54.40 | Hurt |
| Replace trained Judgment Agent with a base Judgment Agent (ablation) | SiriuS Actor-Critic system · PubMedQA | TP Accuracy: 35.00 -> 20.20; 59.80 -> 53.00 | Hurt |
| Replace trained Critic with a base Critic (ablation) | SiriuS Actor-Critic system · PubMedQA | Overall Accuracy: 50.60 -> 50.40; 66.80 -> 64.20 | Mixed |
| Use one shared fine-tuned LLM instead of specialized Actor-Critic agents (ablation) | SiriuS Actor-Critic system · PubMedQA | Overall Accuracy: 50.60 -> 43.60; 66.80 -> 59.60 | Hurt |
Surprising findings
- Self-Correct shows much lower TP accuracy (11.80% for GPT-3.5-Turbo) because it modifies correct answers, a known issue with out-of-the-box self-correction.
- Fine-tuning a single shared LLM for all roles performs notably worse than role-specific fine-tuning.
Limitations
- Experiments use small datasets (e.g., ~100-200 test questions per task) and a single temperature-0 setting, so variance is not assessed.
- Relies on proprietary OpenAI models and fine-tuning API, limiting reproducibility with open models.
- Augmentation uses ground-truth answers in problem-solving settings, so good-trajectory data depends on access to labels.
Experimental setup
Datasets: MMLU GPQA TheoremQA PubMedQA NegotiationArena
Benchmarks: College Physics College Chemistry PubMedQA NegotiationArena resource exchange Ultimatum Seller-Buyer
Metrics: accuracy TP accuracy overall accuracy win rate payoff final selling price
Baselines: Single-Agent STaR CoMM TextGrad Self-Correct Prompt
Builds on: STaR TextGrad CoMM NegotiationArena
Closest papers by meaning
- MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks (ICML 2026 · similarity 0.88)
- The Illusion of Multi-Agent Advantage (NeurIPS 2026 · similarity 0.87)
- MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems (ICLR 2026 · similarity 0.87)
- SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents (NeurIPS 2025 · similarity 0.87)
- Reward Modeling for Multi-Agent Orchestration (NeurIPS 2026 · similarity 0.86)
- MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.