AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

SWE-RM: Execution-free Feedback for Software Engineering Agents

KaShun SHUM, Binyuan Hui, Jiawei Chen, Lei Zhang, X. W., Jiaxi Yang, Yuzhen Huang, Junyang Lin, Junxian He

ICLR 2026 · Reinforcement learning › RL from feedback · approach: Scaling RL with verifiable rewards · 31 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use 2:1 positive-to-negative ratio instead of 1:1 (ablation)SWE-RM verifier (Qwen3-Coder-Flash) · SWE-bench VerifiedAUC: 0.782 -> 0.805Improved
use 2:1 positive-to-negative ratio instead of 1:8 (ablation)SWE-RM verifier (Qwen3-Coder-Flash) · SWE-bench VerifiedECE: 0.349 -> 0.080Improved
use 2:1 ratio instead of 1:1 (ablation)SWE-RM verifier (Qwen3-Coder-Max) · SWE-bench VerifiedRM@32: 70.2% -> 71.0%Mixed
extend verifier context length from 16k to 256k (ablation)SWE-RM verifier · SWE-bench VerifiedRM@32: 66.8% -> 74.4%Improved
extend context length from 16k to 256k (ablation)SWE-RM verifier · SWE-bench Verifiedscore rate: 0.5% -> 100%Improved
use mix-policy data instead of on-policy data (ablation)SWE-RM verifier (Qwen3-Coder-Flash) · SWE-bench VerifiedECE: 0.148 -> 0.033Improved
use mix-policy instead of on-policy data (ablation)SWE-RM verifier (Qwen3-Coder-Max) · SWE-bench VerifiedRM@32: 71.0 -> 70.2Hurt
train on SWE-rebench instead of SWE-smith (ablation)SWE-RM verifier (Qwen3-Coder-Flash) · SWE-bench VerifiedAUC: 0.781 -> 0.814Improved
add SWE-smith to SWE-Gym training data (ablation)SWE-RM verifier (Qwen3-Coder-Flash) · SWE-bench VerifiedECE: 0.087 -> 0.034Improved
use SWE-RM execution-free verifier instead of DeepSWE EF verifierverifier · SWE-bench Verified (Qwen3-Coder-Flash)RM@32: 53.2% -> 62.0%Improved
use hybrid feedback instead of execution-based onlyRL on Qwen3-30B-A3B · SWE-bench Verifiedpass@1: 51.8% -> 54.8%Improved
use hybrid feedback instead of execution-free only (ablation)RL on Qwen3-30B-A3B · Terminal Benchscore: 31.3 -> 32.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy