Research map / Large language models / Agents, code and math
Program Synthesis and Semantic Parsing with Learned Code Idioms
Eui Chul Shin, Miltiadis Allamanis, Marc Brockschmidt, Alex Polozov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add learned code idioms using the best reported configuration | Tree-based decoder · Hearthstone test | Exact match: 0.152 -> 0.197 | Improved |
| Add learned code idioms using the best reported configuration | Tree-based decoder · Hearthstone test | Sentence BLEU: 0.743 -> 0.780 | Improved |
| Add learned code idioms using the best reported configuration | Tree-based decoder · Hearthstone test | Corpus BLEU: 0.723 -> 0.766 | Improved |
| Add learned code idioms ranked by Score_CXE with K=80 | Tree-based decoder · Spider dev | Exact match: 0.395 -> 0.416 | Improved |
| Vary the idiom count and ranking method (Score_Cov or Score_CXE; K=10, 20, 40, or 80) (ablation) | Patois · Hearthstone dev | Exact match: 0.197; 0.151, 0.091, 0.167, 0.197; 0.151, 0.167, 0.182, 0.151 | Mixed |
| Vary the idiom count and ranking method (Score_Cov or Score_CXE; K=10, 20, 40, or 80) (ablation) | Patois · Hearthstone dev | Sentence BLEU: 0.767; 0.781, 0.745, 0.765, 0.780; 0.780, 0.787, 0.773, 0.771 | Mixed |
| Vary the idiom count and ranking method (Score_Cov or Score_CXE; K=10, 20, 40, or 80) (ablation) | Patois · Hearthstone dev | Corpus BLEU: 0.763; 0.785, 0.745, 0.764, 0.774; 0.783, 0.782, 0.770, 0.768 | Mixed |
| Vary the idiom count and ranking method (Score_Cov or Score_CXE; K=10, 20, 40, or 80) (ablation) | Patois · Spider dev | Exact match: 0.395; 0.394, 0.379, 0.395, 0.407; 0.368, 0.382, 0.387, 0.416 | Mixed |
Closest papers by meaning
- Learning to Infer Program Sketches (ICML 2019 · similarity 0.88)
- Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis (ICLR 2018 · similarity 0.87)
- LILO: Learning Interpretable Libraries by Compressing and Documenting Code (ICLR 2024 · similarity 0.87)
- Leveraging Language to Learn Program Abstractions and Search Heuristics (ICML 2021 · similarity 0.87)
- HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesis (NeurIPS 2024 · similarity 0.86)
- Generating Pragmatic Examples to Train Neural Program Synthesizers (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
