Research map / Large language models / Agents, code and math
Discovering Novel LLM Experts via Task-Capability Coevolution
Andrew Dai, Boris Meinardus, Ciaran Regan, Yingtao Tian, Yujin Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AC/DC task force instead of hand-picked experts | 3 expert models · Benchmark Coverage, N=3 | coverage: +3.99 pp avg | Improved |
| Use AC/DC N=8 task force instead of single larger model | Big Model · Benchmark Coverage, N=8 | coverage: +10.19 pp avg | Improved |
| AC/DC N=3 vs GPT-4o | GPT-4o · Benchmark Coverage, N=3 | coverage: -4.52 pp avg | Mixed |
| AC/DC N=8 vs GPT-4o | GPT-4o · Benchmark Coverage, N=8 | coverage: +2.99 pp avg | Improved |
| AC/DC task force vs sampling instruct model repeatedly (control) | Control · Benchmark Coverage, N=3 and N=8 | coverage: +2.45 / +2.04 pp avg (Qwen2 -0.45, -1.04) | Mixed |
| Select with Best-of-N instead of Coverage: AC/DC vs experts | 3 expert models · Benchmark Best-of-N, N=3 | accuracy: +1.34 pp avg (Qwen -1.31, -1.26, -0.49; DeepSeek +11.73) | Mixed |
| AC/DC Best-of-N vs big model | Big Model · Benchmark Best-of-N | accuracy: -3.89 pp (N=3), -0.25 pp (N=8) | Hurt |
| AC/DC Best-of-N vs GPT-4o | GPT-4o · Benchmark Best-of-N | accuracy: -10.82 pp (N=3), -7.17 pp (N=8) | Hurt |
| Add QD (DNS) selection (ablation) | AC/DC · Coverage ablation | coverage: +2.39 pp (N=3), +0.88 pp (N=8) | Improved |
| Add gibberish filter (ablation) | AC/DC · Coverage ablation | coverage: +2.46 pp (N=3), +1.18 pp (N=8) | Improved |
| Add mutation or novelty filtering (ablation) | AC/DC · Coverage ablation | coverage: +0.50-1.16 pp (N=3), +0.37-1.19 pp (N=8) | Improved |
| Add task coevolution vs static synthetic dataset (ablation) | AC/DC · Coverage N=8 | coverage: +3.62 pp | Improved |
Closest papers by meaning
- Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.89)
- Large Language Models Think Too Fast To Explore Effectively (NeurIPS 2025 · similarity 0.89)
- Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence (ICML 2025 · similarity 0.89)
- Language Modeling by Language Models (NeurIPS 2025 · similarity 0.89)
- LLM Augmented LLMs: Expanding Capabilities through Composition (ICLR 2024 · similarity 0.88)
- LLM Circuit Analyses Are Consistent Across Training and Scale (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
