Research map / Large language models / Agents, code and math
Asking What Matters: Reward-Driven Clarification for Software Engineering Tasks
Sanidhya Vijayvargiya, Vijay Viswanathan, Graham Neubig
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Stage 3 answerability reward (ablation) | CLARITI · SWE-Bench Verified underspecified (250 issues) | task success rate (%): 24.4 -> 36.8 | Improved |
| add Stage 4 task relevance reward (ablation) | CLARITI · SWE-Bench Verified underspecified (250 issues) | task success rate (%): 32.0 -> 36.8 | Improved |
| add RL with staged rewards after SFT (ablation) | Qwen3-8B · SWE-Bench Verified underspecified (250 issues) | task success rate (%): 24.2 -> 36.8 | Improved |
| add clarification questions from CLARITI | Seed-OSS-36B agent · SWE-Bench Verified underspecified (250 issues) | task success rate (%): 22.4 -> 36.8 | Improved |
| use CLARITI instead of GPT-5 for clarification | agent pipeline · SWE-Bench Verified underspecified (250 issues) | average number of questions: 5.1 -> 3.0 | Improved |
| use CLARITI instead of GPT-5 | agent pipeline · SWE-Bench Verified underspecified (250 issues) | task success rate (%): 35.6 -> 36.8 | No clear effect |
| use GPT-5 instead of GPT-5 Nano for questions | agent pipeline · SWE-Bench Verified underspecified (250 issues) | task success rate (%): 29.6 -> 35.6 | Improved |
| remove information categories from issue (underspecify) | Seed-OSS-36B agent · SWE-Bench Verified rewrites (700 instances) | task success rate (%): 43.8 -> 23.7 | Hurt |
| increase number of clarification questions | clarification sets · SWE-Bench Verified underspecified (500 issues) | task success rate | No clear effect |
| add Stage 4 relevance weighting: share of Error Information questions | CLARITI · question category distribution | share of questions (%): 10.2% -> 26.4% | Improved |
Closest papers by meaning
- Interactive Agents to Overcome Underspecificity in Software Engineering (ICLR 2026 · similarity 0.85)
- Uncertainty-Aware Clarification in LLM Agents with Information Gain (ICML 2026 · similarity 0.85)
- Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents? (NeurIPS 2026 · similarity 0.85)
- ToolRL: Reward is All Tool Learning Needs (NeurIPS 2025 · similarity 0.83)
- TOM-SWE: User Mental Modeling For Software Engineering Agents (ICML 2026 · similarity 0.82)
- SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
