Research map / Large language models / Agents, code and math
QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Code
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces NeuComBack, an IR-to-assembly benchmark derived from ExeBench and TSVC, and a self-evolving prompt learning method that evolves a single prompt from LLM self-debugging traces, improving DeepSeek-R1's x86_64 and aarch64 assembly correctness and performance over clang-O3.
Problem
Neural compilation (LLM translation of LLVM IR to assembly) lacks dedicated benchmarks and methods that reliably improve functional correctness and performance of LLM-generated assembly.
Approach
Offline, the same LLM acts as a prompt optimizer: it analyzes self-debug trajectories from initially failing but later-corrected programs, extracts error patterns and fixes, and edits a single evolving prompt in mini-batches with an LLM-based review step; the learned prompt is then used for online generation and iterative optimization.
Key results
- On NeuComBack-L2 (x86_64, DeepSeek-R1), learned prompt raised initial-generation ACC from 44% (11/25) to 64% (16/25) and ACC+Perf after 2 optimization rounds from 28% to 56%.
- On aarch64 NeuComBack-L2 test set, ACC rose from 36% to 72% (9/25 to 18/25) with ACC+Perf 8% to 28%.
- Among 16 correct x86_64 programs with learned prompt, 14 (87.5%) surpassed clang-O3, average speedup 1.28x.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the baseline prompt with the self-evolving learned prompt | DeepSeek-R1 · NeuComBack-L1 test, x86_64 | ACC (%): 50.00 (20/40) -> 80.00 (32/40) | Helped |
| Replace the baseline prompt with the self-evolving learned prompt | DeepSeek-R1 · NeuComBack-L2 test, x86_64, initial generation | ACC (%): 44.00 (11/25) -> 64.00 (16/25) | Helped |
| Replace the baseline prompt with the self-evolving learned prompt | DeepSeek-R1 · NeuComBack-L2 test, x86_64, initial generation | ACC+Perf (%): 24.00 (6/25) -> 40.00 (10/25) | Helped |
| Use the learned prompt rather than the baseline prompt throughout 2 optimization rounds | DeepSeek-R1 · NeuComBack-L2 test, x86_64 | ACC+Perf (%): 28.00 (7/25) -> 56.00 (14/25) | Helped |
| Replace the baseline prompt with the learned prompt (ablation) | DeepSeek-R1 · NeuComBack-L2 test, aarch64 | ACC (%): 36.00 (9/25) -> 72.00 (18/25) | Helped |
| Replace the baseline prompt with the learned prompt (ablation) | DeepSeek-R1 · NeuComBack-L2 test, aarch64 | ACC+Perf (%): 8.00 (2/25) -> 28.00 (7/25) | Helped |
| Transfer the prompt learned on NeuComBack-L2 without further learning instead of using the default prompt (ablation) | DeepSeek-R1 · NeuComBack-L1 test, x86_64 | ACC (%): 50.00 (20/40) -> 67.50 (27/40) | Helped |
| Learn the prompt on NeuComBack-L1 rather than transfer the NeuComBack-L2 prompt (ablation) | DeepSeek-R1 · NeuComBack-L1 test, x86_64 | ACC (%): 67.50 (27/40) -> 80.00 (32/40) | Helped |
| Use the self-evolving learned prompt (ablation) | DeepSeek-R1 · NeuComBack-L1 test, x86_64 | Average self-debug rounds: 0.90 -> 0.28 | Helped |
| Use the self-evolving learned prompt (ablation) | DeepSeek-R1 · NeuComBack-L2 test, x86_64 | Average self-debug rounds: 1.09 -> 0.25 | Helped |
| Use the self-evolving learned prompt (ablation) | DeepSeek-R1 · NeuComBack-L2 test, aarch64 | Average self-debug rounds: 1.44 -> 1.22 | Helped |
| Increase the sampling budget to 14× for DeepSeek-V3 under cost control (ablation) | DeepSeek-V3 versus DeepSeek-R1 · NeuComBack-L2 test, 25 cases | Problems solved (ACC): 11/25 -> 17/25 | Helped |
Surprising findings
- A DeepSeek-V3 model given 14x sampling budget solved 17/25 L2 problems, versus 11/25 for a single DeepSeek-R1 run, suggesting cost-matched comparisons change rankings.
- A prompt learned on NeuComBack-L2 transferred to NeuComBack-L1, raising test ACC from 50% to 67.5%.
Limitations
- Benchmark covers only LLVM IR to assembly and is moderate in size; L2 focuses on loop-centric TSVC kernels, limiting coverage of other optimization classes.
- No error bars or statistical significance tests due to LLM API costs; small test sets (25-40 samples).
- Method's cost is high and efficacy may be limited for weaker models; evaluated mainly on DeepSeek-R1.
Experimental setup
Datasets: ExeBench TSVC
Benchmarks: NeuComBack-L1 NeuComBack-L2
Metrics: ACC ACC+Perf average self-debug rounds
Baselines: GPT-4o O3-Mini O1 DeepSeek-V3 clang-O3 Default baseline prompt
Builds on: Automatic prompt optimization chain-of-thought
Closest papers by meaning
- Guess & Sketch: Language Model Guided Transpilation (ICLR 2024 · similarity 0.89)
- REASONING COMPILER: LLM-Guided Optimizations for Efficient Model Serving (NeurIPS 2025 · similarity 0.87)
- SelfCodeAlign: Self-Alignment for Code Generation (NeurIPS 2024 · similarity 0.85)
- ASAP: Assembly-Source Aligned Pseudocode Refinement For Binary Decompilation (NeurIPS 2026 · similarity 0.85)
- SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning (NeurIPS 2025 · similarity 0.85)
- Learning to Compile Programs to Neural Networks (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.