Research map / Large language models / Agents, code and math
Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compiler-R1 trains Qwen2.5-Instruct LLM agents for LLVM pass-sequence auto-tuning with SFT cold-start on a synthesized tool-use dataset followed by outcome-reward RL (GRPO, PPO, REINFORCE++).
Problem
LLM-based compiler auto-tuning lacks high-quality reasoning/tool-use training data and suffers from limited interaction with the compilation environment, leading to poor generalization to unseen programs.
Approach
A two-stage pipeline: SFT on simulated thought-tool-answer trajectories built from synergistic pass pairs and graph-guided sequence search, then on-policy RL with a composite format and outcome (IR instruction count reduction over unoptimized code) reward, letting the agent interact with instrcount and find_best_pass_sequence tools.
Key results
- GRPO-7B achieves an average 8.46% IR instruction count reduction over opt -Oz across seven test suites, versus 4.66% for SFT-Qwen-7B (N=40).
- GRPO-7B reaches a 96.71% task success rate; GRPO-1.5B reaches 83.36%.
- GRPO-7B takes about 26 s per program, compared with 200 s for OpenTuner and 9803 s for CompTuner.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GRPO-based interactive RL training instead of direct SFT pass prediction | Qwen2.5-7B-Instruct compiler tuner · seven test suites (average) | OverOz (%): 4.66 -> 8.46 | Helped |
| Use GRPO-7B instead of OpenTuner | compiler auto-tuning · seven test suites (average) | OverOz (%): 6.09 -> 8.46 | Helped |
| Increase model scale from GRPO-1.5B to GRPO-7B | Compiler-R1 · seven test suites (average) | OverOz (%): 3.87 -> 8.46 | Helped |
| Remove RL fine-tuning and use SFT-only training (ablation) | Qwen-1.5B interactive compiler agent · 335 test programs; repetition penalty 1.05 | task success rate (%): 83.36 -> 26.98 | Hurt |
| Add SFT initialization before GRPO rather than training with RL only (ablation) | Qwen-1.5B interactive compiler agent · 335 test programs; repetition penalty 1.05 | task success rate (%): 22.55 -> 83.36 | Helped |
| Increase repetition penalty from 1.05 to 1.10 (ablation) | Compiler-R1 GRPO-7B · 335 test programs | task success rate (%): 51.92 -> 96.71 | Helped |
| Use AutoPhase features instead of raw LLVM IR as input (ablation) | SFT-Qwen-1.5B · 100 held-out functions; N=40 | OverOrig: 0.4795 -> 0.4837 | No clear effect |
| Use greedy decoding instead of single-shot sampled decoding (ablation) | SFT compiler tuner · Experiment 3 held-out functions; N=1 | OverOrig | Helped |
| Increase SFT sampling budget from N=1 to N=40 (ablation) | SFT-Qwen-7B · seven test suites (average) | OverOz (%): -14.78 -> 4.66 | Helped |
| Use GRPO-3B instead of GRPO-1.5B | Compiler-R1 · seven test suites (average) | OverOz (%): 3.87 -> 5.12 | Helped |
| Use PPO-LSTM instead of PPO-noLSTM (ablation) | AutoPhase · seven test suites (average) | OverOz (%): -50.35 -> 1.24 | Helped |
Surprising findings
- GRPO-3B achieves higher OverOz (5.12%) than GRPO-1.5B despite a much lower success rate (45.56%), because it sometimes skips protocol checks.
- RL-only training without SFT cold start yields very low success rates (22.55% at 1.05 repetition penalty, 8.92% at 1.10), showing SFT initialization is needed.
Limitations
- Gains are measured against opt -Oz on a fixed 124-pass action space in LLVM 10.0.0.
- Success rate depends on repetition penalty tuning, and some models bypass protocol checks.
- The SFT trajectories are simulated rather than collected from live agent interaction.
Experimental setup
Datasets: CompilerGym LLVM datasets (six training datasets) cbench-v1 chstone-v0 mibench-v1 npb-v0 blas opencv tensorflow
Benchmarks: blas cbench chstone mibench npb opencv tensorflow
Metrics: OverOz % IR instruction count reduction task success rate OverOrig time per program
Baselines: opt -Oz OpenTuner GA TPE RIO CompTuner BOCA AutoPhase (PPO-LSTM) AutoPhase (PPO-noLSTM) SFT-only Qwen
Builds on: DeepSeek-R1 PPO GRPO REINFORCE++ AutoPhase CompilerGym
Closest papers by meaning
- R1-Code-Interpreter: LLMs Reason with Code via Supervised and Multi-stage Reinforcement Learning (ICLR 2026 · similarity 0.89)
- RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning (ICML 2025 · similarity 0.89)
- REASONING COMPILER: LLM-Guided Optimizations for Efficient Model Serving (NeurIPS 2025 · similarity 0.88)
- Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization (NeurIPS 2025 · similarity 0.87)
- $\mathcal{B}$-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis (ICLR 2024 · similarity 0.87)
- Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.