Research map / Large language models / Agents, code and math
FeatureBench: Benchmarking Agentic Coding for Complex Feature Development
Qixing Zhou, JiaCheng Zhang, Haiyang Wang, Rui Hao, Jiahe Wang, Minghao Han, Yuxue Yang, Shuzhe Wu, Feiyang Pan, Lue Fan, Dandan Tu, Zhaoxiang Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add interface definitions to the task (ablation) | Gemini-3-Pro-Preview (OpenHands) · FeatureBench Lite | % resolved: 3.3 -> 10.0 | Improved |
| Add interface definitions to the task (ablation) | GPT-5.1-Codex · FeatureBench Lite | % resolved: 16.7 -> 20.0 | Improved |
| Make unit tests visible to the agent (ablation) | Gemini-3-Pro-Preview (OpenHands) · FeatureBench Lite | % resolved: 10.0 -> 60.0 | Improved |
| Make unit tests visible to the agent (ablation) | GPT-5.1-Codex · FeatureBench Lite | % resolved: 20.0 -> 63.3 | Improved |
| Evaluate on FeatureBench instead of SWE-bench Verified | Claude Opus 4.5 · resolved rate | % resolved: 74.40 -> 5.2 | Hurt |
| Use Claude Opus 4.5 instead of Qwen3-Coder under OpenHands | OpenHands · FeatureBench Full | % resolved: 3.5 -> 10.5 | Improved |
| Use Lite subset instead of Full set | all configurations · FeatureBench | ranking | No clear effect |
Closest papers by meaning
- GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging (AAAI 2026 · similarity 0.91)
- RoadmapBench: Evaluating Long-Horizon Agentic Software Development Across Version Upgrades (NeurIPS 2026 · similarity 0.91)
- SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks (NeurIPS 2026 · similarity 0.90)
- SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (NeurIPS 2026 · similarity 0.90)
- ProgramBench: Can Language Models Rebuild Programs From Scratch? (NeurIPS 2026 · similarity 0.89)
- EvoCodeBench: Evaluating Coding Agents in Multi-Turn Iterative Interactions (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
