Research map / Large language models / Agents, code and math
Learning to Compile Programs to Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces neural surrogate compilation, where a hypernetwork (CompNet, a BERT-Tiny encoder with a linear parameter head) maps C program text directly to initial weights for a neural surrogate, which is then finetuned.
Problem
Training neural surrogates of programs is data- and epoch-expensive when trained from scratch per program, while universal neural surrogates (LMs that both read program text and execute it) are costly and inaccurate for resource-constrained deployment.
Approach
Use a hypernetwork that consumes program text and outputs parameters for a fixed covering MLP architecture, so the compiled initialization is a good starting point that can be finetuned cheaply; generation and execution are decoupled, unlike universal neural surrogates.
Key results
- CompNet-initialized surrogates achieve 9.50x geometric mean test-loss improvement over random initialization on ExeStackCPN test programs, versus 1.09x for MAML and 1.08x for pretrained surrogates.
- On ParrotBenchCPN, CompNet achieves 1.91x average improvement over random initialization, with 7.85x on kmeans and 1.01x on invk2j.
- CompNet surrogates reach a target error in 4.3-7.3x fewer epochs and produce color-quantized images with 1.0-1.3x more similar results to ground truth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize surrogates from program text using CompNet instead of random initialization | Neural surrogates on ExeStackCPN test programs · ExeStackCPN test programs | geometric mean test loss improvement over random initialization: 9.50× | Helped |
| Use MAML initialization instead of random initialization | Neural surrogates on ExeStackCPN test programs · ExeStackCPN test programs | geometric mean test loss improvement over random initialization: 1.09× | Helped |
| Use pretrained neural surrogate initialization instead of random initialization | Neural surrogates on ExeStackCPN test programs · ExeStackCPN test programs | geometric mean test loss improvement over random initialization: 1.08× | Helped |
| Initialize surrogates from program text using CompNet instead of random initialization | Neural surrogates on ParrotBenchCPN · ParrotBenchCPN | geometric mean test loss improvement over random initialization: 1.91× | Helped |
| Use MAML initialization instead of random initialization | Neural surrogates on ParrotBenchCPN · ParrotBenchCPN | geometric mean test loss improvement over random initialization: 0.93× | Hurt |
| Use pretrained neural surrogate initialization instead of random initialization | Neural surrogates on ParrotBenchCPN · ParrotBenchCPN | geometric mean test loss improvement over random initialization: 1.05× | Helped |
| Initialize the surrogate from program text using CompNet instead of random initialization | Neural surrogates for kmeans · ParrotBenchCPN kmeans | test loss improvement over random initialization: 7.85× | Helped |
| Use MAML initialization instead of random initialization | Neural surrogates for kmeans · ParrotBenchCPN kmeans | test loss improvement over random initialization: 0.68× | Hurt |
| Use pretrained neural surrogate initialization instead of random initialization | Neural surrogates for fft and sobel · ParrotBenchCPN fft and sobel | test loss improvement over random initialization: 0.61×; 0.85× | Hurt |
| Initialize the surrogate from program text using CompNet instead of random initialization | Neural surrogates for color quantization · Color quantization (5-color palette) | image similarity to exact implementation: 1.0-1.3× | Helped |
| Initialize the surrogate from program text using CompNet instead of random initialization | Neural surrogate training · Neural surrogate training | epochs to target error: 4.3-7.3× fewer epochs | Helped |
Surprising findings
- Pretrained neural surrogates (a single shared initialization) worsen performance at most dataset sizes on ParrotBenchCPN, and MAML worsens performance on average there (0.93x).
- In the zero-shot regime on ExeStackCPN, CompNet improves loss by 84.40x over random initialization.
Limitations
- Covering architecture is a small fixed MLP (9 inputs, 4-4 hidden, 1 output) and programs are restricted to pointer-free numeric C functions with at most 9 inputs and 512 tokens.
- Gains are relative to random initialization; absolute error is only shown to be sufficient for downstream applications.
- Decontamination against ParrotBench required bespoke syntactic filters, and only 4 ParrotBench programs are evaluated.
Experimental setup
Datasets: ExeStack ExeStackCPN ParrotBench The Stack
Benchmarks: ExeStackCPN test programs ParrotBenchCPN (fft, invk2j, kmeans, sobel) color quantization (kmeans application)
Metrics: test MSE loss improvement over random initialization (geomet MSE and SSIM for color quantization epochs to target loss
Baselines: random initialization MAML pretrained neural surrogates
Builds on: BERT hypernetworks MAML Parrot transformation
Closest papers by meaning
- Adaptive Neural Compilation (NeurIPS 2016 · similarity 0.88)
- RobustFill: Neural Program Learning under Noisy I/O (ICML 2017 · similarity 0.87)
- Neuro-Symbolic Program Synthesis (ICLR 2017 · similarity 0.87)
- Execution-Guided Neural Program Synthesis (ICLR 2019 · similarity 0.87)
- QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Code (NeurIPS 2025 · similarity 0.85)
- HYSYNTH: Context-Free LLM Approximation for Guiding Program Synthesis (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.