Research map / Large language models / Agents, code and math
Teaching Large Language Models to Self-Debug
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes self-debugging, where an LLM debugs its own predicted programs via few-shot prompting using execution feedback and explanations, without updating weights.
Problem
LLMs often fail to generate correct programs for complex coding tasks in one attempt, and prior program repair approaches require additional training or external feedback.
Approach
Teaches an LLM to debug its own predicted program via few-shot demonstrations that pair execution traces (unit test results, runtime feedback) with natural-language explanations of the code, enabling iterative self-repair in a training-free manner.
Key results
- Self-debugging with few-shot prompting achieves state-of-the-art results on Spider text-to-SQL and TransCoder code translation among the evaluated settings, per the abstract's claims.
- Self-debugging improves Codex (code-davinci-002) baseline accuracy on MBPP and Spider by several points when given unit test feedback.
Limitations
- Full text provided is the ICLR style template rather than the paper body, so detailed results could not be verified.
- Relies on execution feedback or unit tests that may not be available in all settings.
Experimental setup
Datasets: Spider TransCoder MBPP
Benchmarks: Spider TransCoder MBPP
Metrics: accuracy pass@1
Baselines: Codex greedy decoding CodeT GPT-4 baseline program repair with separate repair model
Builds on: chain-of-thought prompting self-repair
Closest papers by meaning
- NExT: Teaching Large Language Models to Reason about Code Execution (ICML 2024 · similarity 0.90)
- Is Self-Repair a Silver Bullet for Code Generation? (ICLR 2024 · similarity 0.90)
- LeDex: Training LLMs to Better Self-Debug and Explain Code (NeurIPS 2024 · similarity 0.90)
- CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer (ICML 2026 · similarity 0.89)
- BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions (ICLR 2025 · similarity 0.88)
- Precise Debugging Benchmark: Is Your Model Debugging or Regenerating? (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.