Research map / Large language models / Agents, code and math

SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution

NeurIPS 2025 · Large language models › Agents, code and math · approach: Software engineering agents · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SWE-RL, applying GRPO with a rule-based difflib similarity reward on real-world GitHub pull-request data to train Llama-3.3-70B into SWE-bench-solving reasoning model Llama3-SWE-RL-70B, which reaches 41.0% on SWE-bench Verified and shows out-of-domain gains.

Problem

Improving LLM reasoning for real-world software engineering issue solving, where RL with execution feedback is hard to apply and prior training relied on distilled proprietary-model SFT data.

Approach

Use software evolution data (merged PRs with linked issues, code context, and oracle patches) as an RL seed set with a lightweight rule-based reward: -1 for bad format, otherwise difflib SequenceMatcher similarity between predicted and oracle patch, optimized with GRPO.

Key results

• Llama3-SWE-RL-70B resolves 41.0% of SWE-bench Verified with Agentless Mini, best among open models under 100B and comparable to GPT-4o (38.8% with Agentless).
• On out-of-domain benchmarks, the RL model improves over Llama-3.3-70B-Instruct on HumanEval+ (79.9 vs 76.2), CRUXEval-O (75.5 vs 61.9), and MATH strict (73.7 vs 63.2), while the SFT variant declines on average.
• Repair-only oracle setting: RL model reaches 34.8 vs 29.6 for the SFT baseline and 5.4 for base Llama-3.3-70B-Instruct under greedy decoding.

Measured effects reported in the paper

Surprising findings

• RL trained solely on issue-solving data improves out-of-domain tasks such as math and code reasoning, while SFT on the same domain degrades average OOD performance.
• A discrete exact-match reward yields near-zero average reward at the end of training, whereas the continuous similarity reward drives learning.

Limitations

• Sequence-similarity reward does not capture semantic equivalence of alternative functional patches.
• Agentless Mini's localization is simplified to file-path mapping, lacking comprehensive context.
• Pipeline-based structure prevents the model from learning from interactive execution feedback.

Experimental setup

Datasets: GitHub pull requests (24M aggregated, 11M filtered, 273k RL seeds) SWE-bench Verified HumanEval+ BigCodeBench CRUXEval MATH MMLU

Benchmarks: SWE-bench Verified HumanEval+ BigCodeBench-Hard CRUXEval MATH MMLU

Metrics: pass@1 resolve rate correct format rate repair performance macro average accuracy

Baselines: Llama-3.3-70B-Instruct Llama3-SWE-SFT-70B GPT-4o Claude-3.5-Sonnet Lingma-SWE-GPT SWE-Gym SWE-Fixer Agentless

Builds on: GRPO DeepSeek-R1 Agentless Magicoder OSS-Instruct CodeT

Closest papers by meaning

• Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.90)
• R1-Code-Interpreter: LLMs Reason with Code via Supervised and Multi-stage Reinforcement Learning (ICLR 2026 · similarity 0.90)
• ReTool: Reinforcement Learning for Strategic Tool Use in LLMs (ICLR 2026 · similarity 0.90)
• Diversity-Aware Policy Optimization for Large Language Model Reasoning (NeurIPS 2025 · similarity 0.89)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.89)
• RL Tango: Reinforcing Generator and Verifier Together for Language Reasoning (NeurIPS 2025 · similarity 0.89)