Research map / Multi-agent systems and games / LLM agents
daVinci-Dev: Agent-native Mid-training for Software Engineering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes agentic mid-training with two agent-native PR-derived datasets (contextually-native 68.6B tokens and environmentally-native 3.1B tokens) that yields daVinci-Dev models reaching 56.1% (32B) and 58.5% (72B) on SWE-Bench Verified under SWE-Agent.
Problem
Post-training alone leaves a distribution mismatch between static pre-training data and the interactive, feedback-rich environment of software engineering agents; agentic mid-training remains underexplored.
Approach
Construct agent-native mid-training data from GitHub pull requests (bundling issue, relevant base files, summaries and temporal edits into one sample) plus test-verified rollouts from executable Docker environments, and use it as a mid-training stage before SFT on Qwen2.5 base models.
Key results
- daVinci-Dev-72B reaches 58.5% on SWE-Bench Verified under SWE-Agent, versus 48.6% for Kimi-Dev with the same base and scaffold.
- daVinci-Dev-32B reaches 56.1% on SWE-Bench Verified, reported as state-of-the-art among open recipes with agentic scaffolds at 32B despite starting from non-coder Qwen2.5-32B-Base.
- In zero-shot (no SFT) evaluation, adding PR data to environment trajectories raises 72B SWE-V from 47.1% to 54.8%.
Surprising findings
- Training on environmentally-native trajectories alone is insufficient; pairing them with PR data is needed for zero-shot agentic capability (+7.7 points on 72B).
- Mid-training with trajectories improves SFT results even though the same trajectories are used again in SFT (double-dipping still helps).
Limitations
- PR text in the general subset retains developer identifiers, raising privacy and memorization concerns.
- Some results rely on a patched evaluation harness that fixes a few benchmark instances, adding variance.
- Only one base model family and one benchmark (SWE-Bench Verified) are studied for the main results.
Experimental setup
Datasets: GitHub pull requests (general and Python subsets) environmentally-native SWE-agent rollouts SWE-smith trajectories HumanEval EvalPlus
Benchmarks: SWE-Bench Verified HumanEval EvalPlus DS-1000 GPQA-Main SuperGPQA SciBench
Metrics: Pass@1 resolution rate accuracy
Baselines: Kimi-Dev SWE-Agent-LM Openhands-LM SWE-Mirror-LM Skywork-SWE SWE-Dev DeepSWE-Preview FrogBoss SWE-Lego-Qwen3-32B R2EGym-Agent
Builds on: Kimi-Dev SWE-smith SWE-rebench SWE-Agent The Stack v2
Closest papers by meaning
- Kimi-Dev: Agentless Training as Skill Prior for SWE-agents (ICLR 2026 · similarity 0.90)
- SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents (NeurIPS 2025 · similarity 0.88)
- From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents (NeurIPS 2026 · similarity 0.87)
- Toward Training Superintelligent Software Agents through Self-Play SWE-RL (ICML 2026 · similarity 0.87)
- Discovering Hierarchical Software Engineering Agents via Bandit Optimization (ICLR 2026 · similarity 0.86)
- SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineering (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.