Research map / Large language models / Language, safety and interpretability
Synchromesh: Reliable Code Generation from Pre-trained Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CSD and TST together | Codex 175B · Spider | execution accuracy: 56% -> 64% | Helped |
| Apply CSD and TST together | Codex 175B · Spider | valid outputs: 73% -> 85% | Helped |
| Apply CSD | Codex 175B · Vega-Lite NLV | valid outputs: 87% -> 99% | Helped |
| Apply CSD | Codex 175B · Vega-Lite NLV | exact-match accuracy: 39% -> 40% | Helped |
| Apply CSD and TST together | Codex 175B · SMCalFlow | exact-match accuracy: 45% -> 63% | Helped |
| Apply CSD and TST together | Codex 175B · SMCalFlow | valid outputs: 79% -> 99% | Helped |
| Add TST to CSD (ablation) | Codex 175B · Spider | execution accuracy: 61% -> 64% | Helped |
| Add TST to CSD (ablation) | Codex 175B · SMCalFlow | exact-match accuracy: 63% -> 63% | No clear effect |
| Add CSD to TST (ablation) | Codex 175B · SMCalFlow | valid outputs: 90% -> 99% | Helped |
| Use CSD and TST together rather than either alone (ablation) | Codex 175B · Spider and SMCalFlow | accuracy | Mixed |
| Use TST to retrieve examples by target-program similarity (ablation) | GPT-3 and Codex · SMCalFlow | exact-match accuracy | Helped |
| Use CSD to constrain token generation to valid continuations (ablation) | GPT-3 and Codex · Vega-Lite NLV | valid outputs: 55% -> 100% | Helped |
Closest papers by meaning
- CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer (ICML 2026 · similarity 0.88)
- AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data (NeurIPS 2024 · similarity 0.87)
- ChatGPT-Generated Code Assignment Detection Using Perplexity of Large Language Models (Student Abstract) (AAAI 2024 · similarity 0.87)
- AST-T5: Structure-Aware Pretraining for Code Generation and Understanding (ICML 2024 · similarity 0.87)
- SelfCodeAlign: Self-Alignment for Code Generation (NeurIPS 2024 · similarity 0.87)
- ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.