Research map / Multi-agent systems and games / LLM agents
Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DEI, a meta-module that ensembles diverse SWE agents' patches and reranks them with an LLM code-review committee, raising SWE-Bench Lite resolve rates substantially.
Problem
Individual SWE agents resolve different sets of GitHub issues despite similar overall resolve rates, so no single agent is best; the paper asks how to harness this diversity to select the correct patch among candidates.
Approach
A meta-policy that treats existing SWE agent frameworks as an ensemble and uses an LLM committee to explain (issue, context, location, patch, conflicts) and score each candidate patch 1-10, averaging multiple votes to rerank candidates; it is agnostic to the underlying agent framework.
Key results
- DEI-Open committee of 4 open-source agents raises resolve rate from 27.3% (best individual, Agentless) to 34.3% on SWE-Bench Lite.
- DEI-1 committee of top agents reaches 55.0% resolve rate, ranking first on the SWE-Bench Lite leaderboard at time of submission.
- An oracle selector over 10 different agents (avg 26.6%) could solve 54.3% of issues, showing large diversity headroom.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DEI to rerank candidates from 10 different agents rather than uniform/random selection | SWE agent committee · SWE-Bench Lite, 10 Agents (k=10) | resolve rate: 26.6 -> 35.7 | Helped |
| Use DEI to rerank 10 Agentless runs rather than uniform/random selection | Agentless · SWE-Bench Lite, 10 Runs from Agentless (k=10) | resolve rate: 20.4 -> 26.0 | Helped |
| Use DEI to rerank 10 Aider runs rather than uniform/random selection | Aider · SWE-Bench Lite, 10 Runs from Aider (k=10) | resolve rate: 21.7 -> 24.7 | Helped |
| Use DEI to rerank 10 Moatless runs rather than uniform/random selection | Moatless · SWE-Bench Lite, 10 Runs from Moatless (k=10) | resolve rate: 15.9 -> 26.3 | Helped |
| Add explanations before scoring patches (ablation) | DEI · SWE-Bench Lite, Open Agents | resolve rate: 32.3 -> 34.6 | Helped |
| Add explanations before scoring patches (ablation) | DEI with Agentless candidates · SWE-Bench Lite, Agentless | resolve rate: 23.0 -> 26.0 | Helped |
| Add explanations before scoring patches (ablation) | DEI with Aider candidates · SWE-Bench Lite, Aider | resolve rate: 23.3 -> 24.6 | Helped |
| Add explanations before scoring patches (ablation) | DEI with Moatless candidates · SWE-Bench Lite, Moatless | resolve rate: 25.3 -> 25.6 | Helped |
| Use more votes to average DEI patch scores (ablation) | DEI · SWE-Bench Lite | resolve rate | Helped |
Surprising findings
- Different agents with similar resolve rates solve very different issue sets, and different agents are more diverse than repeated runs of one agent.
- Explanation generation before scoring gives only slight gains (e.g., 26.0 vs 23.0 for Agentless candidates).
Limitations
- Reranker is a single LLM prompt with rule-based rubric; gains plateau and can drop as the number of candidates grows.
- Evaluated only on SWE-Bench Lite (300 instances) with GPT-4o as backbone.
- Top leaderboard entries depend on closed-source agents and models, limiting reproducibility.
Experimental setup
Datasets: SWE-Bench Lite
Benchmarks: SWE-Bench Lite
Metrics: resolve rate Union@k Intersect@k Average@k n@k
Baselines: Agentless Moatless Tools Aider OpenDevin + CodeAct AutoCodeRover CodeR MASAI SIMA Cosine Genie CodeStory Aide
Builds on: SWE-bench SWE-agent Agentless Moatless Tools CodeAct
Closest papers by meaning
- Discovering Hierarchical Software Engineering Agents via Bandit Optimization (ICLR 2026 · similarity 0.87)
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (NeurIPS 2024 · similarity 0.87)
- SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement (ICLR 2025 · similarity 0.87)
- CoMind: Towards Community-Driven Agents for Machine Learning Engineering (ICLR 2026 · similarity 0.86)
- Interactive Agents to Overcome Underspecificity in Software Engineering (ICLR 2026 · similarity 0.86)
- ProSoftArena: Benchmarking Hierarchical Capabilities of Multi-modal Agents in Professional Software Environments (CVPR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.