Research map / Multi-agent systems and games / LLM agents

AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AutoScientists, a decentralized self-organizing team of LLM coding agents that critique proposals, form teams around hypotheses, and share failures, evaluated on BioML-Bench, GPT nanochat training optimization, and ProteinGym against Autoresearch and other agents.

Problem

Long-running scientific experimentation needs parallel exploration of competing hypotheses, adaptation as evidence shifts, and memory of failed directions, which single-thread or centrally planned agents handle poorly.

Approach

Agents act on a shared experimental state without a central orchestrator: they post proposals to a forum where peers critique them before compute is spent, self-organize into teams via discussion rounds, reorganize when teams stagnate, and share successes and dead ends across teams.

Key results

• On BioML-Bench (24 tasks), AutoScientists reaches mean leaderboard percentile 74.40% vs 66.07% for Autoresearch (+8.33 points).
• On GPT nanochat from the Autoresearch baseline, AutoScientists reaches val_bpb ~0.978 in 34 experiments vs 65 for Autoresearch (1.9x fewer); from its champion, it accepts 7 improvements to 0.9730 while Autoresearch accepts 0 over 100 experiments.
• The AutoScientists-Kermut recipe raises ProteinGym average Spearman from 0.657 (Kermut) to 0.700 across 217 assays, and ACE2-Spike from 0.747 to 0.840 on the development assay.

Surprising findings

• Single-agent Autoresearch accepted zero improvements over 100 experiments from the AutoScientists champion, while the multi-agent system found seven.
• AutoScientists uses more LLM tokens than Autoresearch, though within the same order of magnitude, and is not designed to be LLM-call efficient.

Limitations

• Not designed to be more LLM-call efficient than single-agent baselines; uses more tokens.
• Evaluated with one H100 per task so GPU experiments run sequentially; scaling with more GPUs untested.
• Number of agents is fixed before the run; dynamic team sizing is future work. Kermut ProteinGym result trades off MSE (slightly higher).

Experimental setup

Datasets: BioML-Bench ProteinGym TDC Polaris Open Problems Kaggle biomedical imaging tasks nanochat training data

Benchmarks: BioML-Bench GPT nanochat val_bpb ProteinGym supervised substitution (217 DMS assays)

Metrics: leaderboard percentile medal rate val_bpb Spearman correlation MSE AUROC Pearson correlation odds ratio

Baselines: Autoresearch Biomni AIDE MLAgentBench STELLA Reference ProteinNPT MSA Transformer Embeddings ESM-1v Embeddings Tranception Embeddings Kermut

Builds on: Autoresearch AIDE Kermut BioML-Bench

Closest papers by meaning

• AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite (ICLR 2026 · similarity 0.87)
• From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking (ICLR 2026 · similarity 0.87)
• AI-Researcher: Autonomous Scientific Innovation (NeurIPS 2025 · similarity 0.87)
• CycleResearcher: Improving Automated Research via Automated Review (ICLR 2025 · similarity 0.86)
• SciResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery (NeurIPS 2026 · similarity 0.86)
• BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments (ICLR 2025 · similarity 0.86)