Research map / Large language models / Agents, code and math

DPIAgent: Divide, Protocol, Isolate for Agentic Reproduction Test Generation

NeurIPS 2026 · Large language models › Agents, code and math · approach: Software engineering agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DPIAgent, a phased agent (divide exploration and test writing, protocol handoff summary, isolated toolsets) for reproduction test generation, reaching 81.76% success on SWT-Bench Verified with GPT-5 and 86.17% with test selection.

Problem

Reproduction test generation (writing a fail-to-pass test capturing a reported bug) is treated by existing agents as a monolithic loop, which creates compound objectives and goal drift.

Approach

Structure the agent loop into two phases (defect exploration and test generation) separated by a structured handoff summary, with phase-specific toolsets, while keeping a full agent loop within each phase; add e-Otter++-style test selection using surrogate patches.

Key results

• DPIAgent with DPI alone reaches 81.76% success rate on SWT-Bench Verified with GPT-5, versus 72.52% for SWE-Agent and 79.80% for OpenHands.
• On GPT-5-Mini, DPIAgent reaches 74.28% success, outperforming the strongest baseline OpenHands (62.40%) by 11.88 points.
• Adding test selection (k=6, N=3) raises GPT-5 success to 86.17%, and Claude-Opus-4.5 to 81.38%.

Surprising findings

• Adding DPI structure with only standard SWE-Agent tools yields larger gains than adding the graph search tool or injecting the summary as a prompt alone, suggesting the executable phase boundary is the main source of gain.
• Gains from test selection are larger for weaker models (GPT-5-Mini +8.04) than for Claude-Opus-4.5 (+2.40), despite Claude's higher pass@k ceiling.

Limitations

• Evaluated only on SWT-Bench Verified with three backbones, all closed-source.
• Test selection relies on surrogate patches generated by SWE-Agent rather than the golden patch.
• Baseline OpenHands results partly taken from the public leaderboard and partly re-evaluated, complicating comparison.

Experimental setup

Datasets: SWT-Bench Verified SWE-Bench

Benchmarks: SWT-Bench Verified

Metrics: success rate change coverage mean coverage TDD coverage pass rate vs steps pre-feedback Ready count

Baselines: SWE-Agent Mini-SWE-Agent OpenHands TraeAgent Aider Terminus-2 Claude Code e-Otter++

Builds on: SWE-Agent e-Otter++ Otter

Closest papers by meaning

• SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents (NeurIPS 2024 · similarity 0.86)
• TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark (ICLR 2025 · similarity 0.86)
• DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents (NeurIPS 2026 · similarity 0.85)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems (ICLR 2026 · similarity 0.85)
• TAI3: Testing Agent Integrity in Interpreting User Intent (NeurIPS 2025 · similarity 0.85)
• Property-Guided LLM Program Synthesis for Planning (NeurIPS 2026 · similarity 0.84)