Research map / Multi-agent systems and games / LLM agents

From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes P2T, which uses a reference patch as privileged information to distill a process graph and curate shorter, grounded SWE-agent SFT trajectories from blinded teacher rollouts, reporting gains over outcome-filtered SFT with 1.8k SWE-Gym instances.

Problem

Outcome-filtered rejection sampling for SFT of SWE agents gives no per-step or length signal, retaining trajectories with ungrounded leaps, redundant loops, and discarding instances where the teacher fails.

Approach

Use the developer reference patch as privileged information: a reverse phase distills it into a latent process graph of contextual facts and milestones, and a forward phase commits the shortest blinded teacher segments that make grounded progress on that graph, under a bi-objective (effectiveness, length) rule.

Key results

• On SWE-bench Verified, P2T (full) lifts Pass@1 by up to +10.8 points over test-pass rejection sampling for Qwen2.5-C-32B under Qwen3-C-480B (50.4% vs 39.6%).
• P2T cuts per-instance inference cost by about 15% (e.g., $0.92 to $0.78) while improving Pass@1.
• Removing the groundedness check drops Pass@1 from 50.4% to 43.2% on Qwen2.5-C-32B.

Surprising findings

• Blinded reference solver resolve rate rises from 29-35% to 94-97% as process graph nodes are progressively disclosed, while the oracle patch is never shown.
• About 70% of SWE-Gym instances contribute no supervision under outcome-filtered SFT because the teacher never produces a passing patch.

Limitations

• Relies on the availability of a reference patch and a passing test suite for each instance, restricting to instances with working environments (1.8k of 2.4k).
• Greedy segment-wise commits give no global optimality guarantee over the trajectory space.
• Evaluated only on Python SWE-Gym data and two student backbones; the LLM judge and proposer-critic components depend on teacher quality.

Experimental setup

Datasets: SWE-Gym

Benchmarks: SWE-bench Verified SWE-bench Lite

Metrics: Pass@1 inference cost (US$) interaction length redundant exploration rate

Baselines: Test-pass rejection sampling (SWE-Gym) SWE-Lego

Builds on: SWE-Gym OpenHands SWE-Lego ReAct

Closest papers by meaning

• daVinci-Dev: Agent-native Mid-training for Software Engineering (ICML 2026 · similarity 0.87)
• Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents (ICLR 2026 · similarity 0.87)
• SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (NeurIPS 2026 · similarity 0.87)
• SWE-smith: Scaling Data for Software Engineering Agents (NeurIPS 2025 · similarity 0.86)
• OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories (NeurIPS 2026 · similarity 0.86)
• SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents (NeurIPS 2025 · similarity 0.86)