Research map / Architectures and efficiency / RNNs and normalization
Static Prediction of Runtime Errors by Learning to Execute Programs with External Resource Descriptions
David Bieber, Rishab Goel, Dan Zheng, Hugo Larochelle, Daniel Tarlow
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an interpreter-inspired IPA-GNN instead of a generic Transformer | Runtime error prediction · Python Runtime Errors balanced test | accuracy: 63.6 -> 68.3 | Improved |
| Add external resource descriptions as docstrings (ablation) | GGNN, Transformer, and LSTM · Python Runtime Errors balanced test | accuracy: 62.8 -> 68.3; 63.6 -> 67.3; 66.1 -> 68.1 | Improved |
| Add a resource description as a docstring (ablation) | IPA-GNN · Python Runtime Errors balanced test | weighted error F1: 53.8 -> 62.2 | Improved |
| Add a resource description as a docstring (ablation) | Exception IPA-GNN · Python Runtime Errors balanced test | weighted error F1: 53.3 -> 63.5 | Improved |
| Incorporate the resource description directly with cross-attention rather than as a docstring (ablation) | Exception IPA-GNN · Python Runtime Errors balanced test | accuracy: 71.6 -> 73.8 | Improved |
| Incorporate the resource description directly with cross-attention rather than as a docstring (ablation) | IPA-GNN · Python Runtime Errors balanced test | accuracy: 71.4 -> 72.0 | No clear effect |
| Use FiLM instead of docstring resource input (ablation) | Exception IPA-GNN · Python Runtime Errors balanced test | weighted error F1: 63.5 -> 59.8 | Hurt |
| Add exception-handling transitions to the IPA-GNN (ablation) | IPA-GNN · Python Runtime Errors balanced test | accuracy: 68.3 -> 68.7 | No clear effect |
| Add a resource description (ablation) | Local MIL Transformer · Python Runtime Errors balanced test | localization accuracy: 33.0 -> 48.9 | Improved |
| Add a resource description (ablation) | Global MIL Transformer · Python Runtime Errors balanced test | localization accuracy: 48.2 -> 48.8 | No clear effect |
| Model exception handling and use execution-derived exception provenance to localize errors without location labels | Exception IPA-GNN · Python Runtime Errors balanced test | localization accuracy: 48.2 -> 50.8 | Improved |
| Add cross-attention resource descriptions (ablation) | Exception IPA-GNN · Python Runtime Errors balanced test | localization accuracy: 50.8 -> 68.8 | Improved |
Closest papers by meaning
- NExT: Teaching Large Language Models to Reason about Code Execution (ICML 2024 · similarity 0.86)
- BERT-Based Code Learning for Exception Localization and Type Prediction (AAAI 2025 · similarity 0.84)
- Teaching Large Language Models to Self-Debug (ICLR 2024 · similarity 0.84)
- LeDex: Training LLMs to Better Self-Debug and Explain Code (NeurIPS 2024 · similarity 0.83)
- SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads? (ICML 2026 · similarity 0.83)
- kGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
