Research map / Large language models / Agents, code and math
Co-PatcheR: Collaborative Software Patching with Component-specific Small Reasoning Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Co-PatcheR, a collaborative software patching system using three small component-specific reasoning models (Loc-Gen, Val-assert, Val-no-assert) distilled from teacher LLMs with rejection-sampling-filtered data, reaching 46% resolved on SWE-bench-Verified with 3x14B models.
Problem
Training small models for end-to-end software patching struggles because localization, patch generation, and patch validation have different workflows and expertise needs; prior specialized models of 70B+ size reach only ~30-41% on SWE-bench-Verified.
Approach
Decompose patching into component-specific small reasoning models (one model for localization and generation with critique data, two PoC-generation models with and without assertions plus majority-vote selection), trained by SFT on teacher reasoning traces filtered by answer correctness via rejection sampling.
Key results
- Co-PatcheR (3x14B, PatchPilot scaffold, 60 candidates) reaches 46.00% resolved on SWE-bench-Verified with 6K training data, versus 41.00% for SWE-RL-70B with 11M training data.
- Co-PatcheR-Loc achieves localization comparable to GPT-4o, with o4-mini and Claude-3.7-Sonnet marginally higher.
- Rejection-sampling filtering of wrong-answer reasoning traces improves localization and generation over unfiltered data (Loc-NoFilter, Gen-NoFilter).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use component-specific collaborative models instead of one end-to-end model | Co-PatcheR · SWE-bench-Verified | resolved rate: 41% -> 46% | Helped |
| Use Claude-3.7-Sonnet reasoning instead of Claude-3.5-Sonnet non-reasoning | OpenHands · SWE-bench-Verified | resolved rate: 7% | Helped |
| Add rejection sampling to filter incorrect training answers (ablation) | Co-PatcheR-Loc · SWE-bench-Verified localization | localization accuracy | Helped |
| Add critique training to patch generation (ablation) | Co-PatcheR-Gen · SWE-bench-Verified | pass@1 resolved rate | Helped |
| Add rejection sampling to filter incorrect generation training answers (ablation) | Co-PatcheR-Gen · SWE-bench-Verified | pass@1 resolved rate | Helped |
| Use both Val-assert and Val-no-assert models instead of only Val-no-assert (ablation) | Co-PatcheR validation · SWE-bench-Verified validation candidates | resolved rate | Helped |
| Add filtering to Val-no-assert training data (ablation) | Val-no-assert · SWE-bench-Verified validation candidates | resolved rate | Helped |
| Add diverse reasoning data from Claude-3.7-Sonnet and o4-mini (ablation) | Val-no-assert · SWE-bench-Verified validation candidates | resolved rate | Helped |
| Use PoC tests together with functionality tests (ablation) | Co-PatcheR validation · SWE-bench-Verified validation candidates | resolved rate | Helped |
| Train one model for localization and generation instead of separate models (ablation) | Co-PatcheR · SWE-bench-Verified | performance | No clear effect |
| Increase training data from 500 to 1K samples (ablation) | Co-PatcheR-Gen · SWE-bench-Verified | pass@1 resolved rate | Helped |
| Increase generation model size from 14B to 32B (ablation) | Co-PatcheR-Gen · SWE-bench-Verified | pass@1 resolved rate | Helped |
Surprising findings
- Training on reasoning data with wrong final answers hurts performance, contrary to S1 and Sky-T1 findings on math.
- Rationalization (providing ground-truth patch to teacher) degraded patch generation due to hallucinated identifiers.
Limitations
- No RL training explored; reward design for RL left to future work.
- Uses a simplified agent scaffold without complex tool calls.
- Validation is the bottleneck: with many candidates, selecting the correct patch remains hard.
Experimental setup
Datasets: SWE-bench training set SWE-Gym
Benchmarks: SWE-bench-Verified
Metrics: resolved rate (best@k) top@5 file and line localization accuracy pass@1 resolved rate pass@K inference wall-clock time tokens generated
Baselines: Claude-3.5-Sonnet Claude-3.7-Sonnet OpenHands Agentless PatchPilot SWE-RL-70B SWE-Fixer-72B SoRFT-32B Lingma-SWE-GPT-72B GPT-4o o4-mini Qwen-2.5-Coder-14B
Builds on: PatchPilot Agentless S1 SWE-Gym PPO GRPO
Closest papers by meaning
- PatchPilot: A Cost-Efficient Software Engineering Agent with Early Attempts on Formal Verification (ICML 2025 · similarity 0.90)
- Scratchpad Patching: Decoupling Compute from Patch Size in Byte-Level Language Models (NeurIPS 2026 · similarity 0.86)
- OrcaLoca: An LLM Agent Framework for Software Issue Localization (ICML 2025 · similarity 0.85)
- SWE-bench Goes Live! (NeurIPS 2025 · similarity 0.84)
- SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents (NeurIPS 2024 · similarity 0.84)
- MigGPT: Harnessing Large Language Models for Automated Migration of Out-of-Tree Linux Kernel Patches Across Versions (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.