Research map / Large language models / Agents, code and math
InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Enable up to 10 turns of interaction and execution feedback instead of single-turn generation | LLMs on InterCode-SQL · InterCode-SQL | Success Rate: 9.1% -> 73.7% | Helped |
| Enable up to 10 turns of interaction and execution feedback instead of single-turn generation | LLMs on InterCode-Bash · InterCode-Bash | Success Rate: 34.5 -> 46.5 | Helped |
| Use ReAct rather than Try Again prompting | gpt-3.5-turbo · InterCode-SQL | Success Rate: 47.3 -> 58.7 | Helped |
| Use ReAct rather than Try Again prompting | gpt-3.5-turbo · InterCode-Bash | Success Rate: 46.5 -> 20.5 | Hurt |
| Use Plan & Solve rather than Try Again prompting | gpt-3.5-turbo · InterCode-SQL | Success Rate: 47.3 -> 49.1 | Helped |
| Use Plan & Solve rather than Try Again prompting | gpt-3.5-turbo · InterCode-Bash | Success Rate: 46.5 -> 28.0 | Hurt |
| Add the full relevant database schema to the prompt (ablation) | Single-turn models · InterCode-SQL | Success Rate: 10.5 -> 67.9 | Helped |
| Add the full relevant database schema to the prompt (ablation) | Try Again models · InterCode-SQL | Success Rate: 47.3 -> 72.8 | Helped |
Closest papers by meaning
- RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning (ICML 2025 · similarity 0.87)
- StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning (ICML 2026 · similarity 0.87)
- SelfCodeAlign: Self-Alignment for Code Generation (NeurIPS 2024 · similarity 0.87)
- R1-Code-Interpreter: LLMs Reason with Code via Supervised and Multi-stage Reinforcement Learning (ICLR 2026 · similarity 0.87)
- ConvCodeWorld: Benchmarking Conversational Code Generation in Reproducible Feedback Environments (ICLR 2025 · similarity 0.86)
- Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.