Research map / Large language models / Agents, code and math
Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SED debugger on top of synthesizer | LGRL-GD · Karel test | generalization error: 39.00 -> 16.12 | Helped |
| add SED debugger on top of synthesizer | EGNPS · Karel test | generalization error: 12.64 -> 10.48 | Helped |
| add SED to the 15-model EGNPS ensemble | EGNPS ensemble · Karel test | generalization error: 8.32 -> 7.78 | Helped |
| use SED instead of standard beam search at equal expanded programs | LGRL · Karel test | generalization error: 17.00 -> 15.40 | Helped |
| add fine-tuning of debugger on synthesizer outputs (ablation) | SED · Karel test | generalization error: 18.56 -> 16.12 (LGRL-GD, no TraceEmbed) | Helped |
| add TraceEmbed (execution trace embedding) to debugger (ablation) | SED, fine-tuned · Karel test | generalization error: 16.12 -> 16.12 (LGRL-GD); 14.32 -> 14.28 (LGRL); 10.20 -> 10.48 (EGNPS) | No clear effect |
| add TraceEmbed to debugger without fine-tuning (ablation) | SED · Karel test | generalization error: 18.56 -> 18.68 (LGRL-GD); 14.88 -> 14.60 (LGRL); 10.68 -> 10.68 (EGNPS) | Mixed |
| enlarge hidden sizes instead of adding TraceEmbed (ablation) | SED without TraceEmbed · Karel test, LGRL | generalization error: 15.88 vs 14.88 (no finetune), 16.64 vs 14.32 (finetune) | Hurt |
| use best-first search instead of greedy search over edits (ablation) | SED · Karel test | generalization error | Helped |
| train debugger on 1-5 mutations instead of 1-3 (ablation) | debugger · mutation benchmark | repair accuracy | Mixed |
Closest papers by meaning
- Execution-Guided Neural Program Synthesis (ICLR 2019 · similarity 0.89)
- Write, Execute, Assess: Program Synthesis with a REPL (NeurIPS 2019 · similarity 0.89)
- Improving Neural Program Synthesis with Inferred Execution Traces (NeurIPS 2018 · similarity 0.89)
- Latent Execution for Neural Program Synthesis Beyond Domain-Specific Languages (NeurIPS 2021 · similarity 0.88)
- ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis (ICLR 2024 · similarity 0.85)
- HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesis (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.