Research map / Multi-agent systems and games / LLM agents
Training Software Engineering Agents and Verifiers with SWE-Gym
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SWE-Gym, an executable training environment of 2,438 real-world Python GitHub-issue tasks, and uses rejection-sampling fine-tuning of Qwen2.5-Coder agents plus trained verifiers to reach 32.0% on SWE-Bench Verified with open weights.
Problem
Software engineering agents rely on proprietary LMs because open models lack training environments with executable runtimes, reproducible unit tests, and real task instructions for learning-based post-training.
Approach
Builds a manually configured, executable, test-verified environment from real GitHub pull requests across 11 Python repositories, then uses sampled successful agent trajectories (rejection sampling) to fine-tune open-weight LMs as agents and to train outcome-reward verifiers for best-of-k selection at inference.
Key results
- Fine-tuning Qwen2.5-Coder-32B on 491 SWE-Gym trajectories raises resolve rate from 7.0% to 20.6% on SWE-Bench Verified and from 3.0% to 15.3% on Lite.
- A trained 32B outcome verifier with best-of-16 selection reaches 32.0% on SWE-Bench Verified, a new open-weight state of the art at the time.
- With MoatlessTools, self-improvement via iterative rejection sampling raises the 7B model from 7.0% to 10.0% on SWE-Bench Lite; the 32B verifier lifts it from 19.7% to 26.3%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune OpenHands on 491 SWE-Gym trajectories | Qwen-2.5-Coder-Instruct 32B · SWE-Bench Lite | resolve rate: 3.0% → 15.3% | Helped |
| Fine-tune OpenHands on 491 SWE-Gym trajectories | Qwen-2.5-Coder-Instruct 32B · SWE-Bench Verified | resolve rate: 7.0% → 20.6% | Helped |
| Fine-tune on successful trajectories sampled from the policy being updated, mixed with 491 off-policy trajectories (ablation) | OpenHands Qwen-2.5-Coder-Instruct 32B · SWE-Bench Lite | resolve rate: 15.3 to 8.7% | Hurt |
| Add a first online rejection-sampling iteration (ablation) | MoatlessTools Qwen-2.5-Coder-Instruct 7B · SWE-Bench Lite | resolve rate: 7.0% to 9.0% | Helped |
| Add a second online rejection-sampling iteration (ablation) | MoatlessTools Qwen-2.5-Coder-Instruct 7B · SWE-Bench Lite | resolve rate: 9.0% to 10.0% | Helped |
| Add a second online rejection-sampling iteration (ablation) | MoatlessTools Qwen-2.5-Coder-Instruct 32B · SWE-Bench Lite | resolve rate: 19.7% to 19.7% | No clear effect |
| Use a verifier to select among sampled agent trajectories | OpenHands Qwen2.5-Coder-Instruct 32B · SWE-Bench Verified | resolve rate: 20.6@1 to 29.8@8, and up to 32.0@16 | Helped |
| Fine-tune the verifier with LoRA instead of full parameters (ablation) | OpenHands verifier Qwen2.5-Coder-Instruct 32B · SWE-Bench Verified | Best@8 resolve rate: 29.8@8; 27.2@8 | Helped |
| Train the verifier with a mixture of on-policy and off-policy trajectories instead of only one source (ablation) | OpenHands verifier Qwen2.5-Coder-Instruct 32B · SWE-Bench Verified | Best@8 resolve rate: 27@8 | Helped |
| Use a 32B verifier rather than a 7B verifier to rank 8 trajectories (ablation) | MoatlessTools policy on SWE-Bench Lite · SWE-Bench Lite | resolve rate: 19.7 to 26.3% | Helped |
| Cap selected successful trajectories at two per task rather than use no cap (ablation) | MoatlessTools Qwen-2.5-Coder-Instruct 7B · SWE-Bench Lite | resolve rate: 9.3 to 9.7 | Helped |
| Increase the number of randomly sampled training trajectories (ablation) | OpenHands policy · SWE-Bench Verified | resolve rate | Helped |
Surprising findings
- Self-improvement via fine-tuning on on-policy trajectories dropped resolve rate on SWE-Bench Lite from 15.3% to 8.7%.
- LoRA verifier training outperformed full-parameter fine-tuning (29.8@8 vs 27.2@8).
Limitations
- Environment diversity limited to 11 Python repositories and one language.
- Self-improvement via rejection sampling gives modest gains and fails for the general-purpose OpenHands scaffold.
- Verifier gap between Pass@k and Best@k indicates imperfect reward modeling.
Experimental setup
Datasets: SWE-Gym SWE-Bench training split
Benchmarks: SWE-Bench Verified SWE-Bench Lite
Metrics: resolve rate empty patch rate stuck in loop rate pass@k best@k
Baselines: SWE-Bench Lite zero-shot Qwen2.5-Coder-Instruct Lingma SWE-GPT SWE-Llama SWE-Gym zero-shot base model
Builds on: SWE-Bench OpenHands MoatlessTools rejection sampling fine-tuning
Closest papers by meaning
- SWE-smith: Scaling Data for Software Engineering Agents (NeurIPS 2025 · similarity 0.90)
- Hybrid-Gym: Training Coding Agents to Generalize Across Tasks (ICML 2026 · similarity 0.90)
- SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents (NeurIPS 2025 · similarity 0.89)
- Kimi-Dev: Agentless Training as Skill Prior for SWE-agents (ICLR 2026 · similarity 0.89)
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (NeurIPS 2024 · similarity 0.88)
- SWE-bench: Can Language Models Resolve Real-world Github Issues? (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.