Research map / Reinforcement learning / RL from feedback
Kevin: Multi-Turn RL for Generating CUDA Kernels
Carlo Baronio, Pietro Marsella, Ben Pan, Simon Guo, Silas Alberti
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with multi-turn RL instead of single-turn RL (ablation) | QwQ-32B (Kevin) · KernelBench eval, 16 traj x 8 turns | best@16 speedup: 0.85x -> 1.10x | Improved |
| RL training (multi-turn) on top of base model | QwQ-32B · KernelBench eval | correctness best@16: 56% -> 82% | Improved |
| more serial refinement turns at inference (16x8 vs 128x1) under fixed 128-kernel budget (ablation) | Multi-Turn RL model · KernelBench eval | pass@128 speedup: 0.65x -> 1.10x | Improved |
| single-turn model with 128 parallel single-turn trajectories vs multi-turn model (ablation) | Single-Turn RL · KernelBench eval, 128 traj x 1 turn | pass@128 speedup: 0.70x vs 0.65x | Mixed |
| discounted sum reward with gamma 0.4 vs max aggregation or gamma 0.8 (ablation) | Multi-turn RL reward · KernelBench, 16 traj x 8 turns | scaling with refinement turns | Improved |
| add KL penalty (0.001 or 0.01) to GRPO loss (ablation) | GRPO training · KernelBench training reward | training reward | Hurt |
| add constant length normalization and gradient norm clipping of 0.05 (ablation) | GRPO training · KernelBench training | training stability: junk onset delayed until step 100 | Improved |
| add a length penalty on the response (ablation) | RL training (R1-Distill-Qwen-14B) · KernelBench training | training behavior | Hurt |
| set correctness weight to 1.0 or 0 in score (ablation) | single-turn GRPO · KernelBench training | training reward | Hurt |
| reward intermediate objectives (compile or execute) (ablation) | RL training · KernelBench training | kernel correctness | Hurt |
| use 64 parallel trajectories instead of 16 during multi-turn training (ablation) | Kevin · KernelBench eval | best@16 correctness / performance | No clear effect |
| use a stronger base model (QwQ-32B instead of R1-Distill-Qwen-7B) (ablation) | RL training · KernelBench training | reward hacking and learning | Improved |
Closest papers by meaning
- DRTriton: Large-Scale Synthetic Data Driven Reinforcement Learning for Triton Kernel Generation (NeurIPS 2026 · similarity 0.87)
- StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning (ICML 2026 · similarity 0.87)
- CUDABench: Benchmarking LLMs for Text-to-CUDA Generation (NeurIPS 2026 · similarity 0.86)
- EGG: An Expert-Guided Agent Framework for Kernel Generation (ICML 2026 · similarity 0.85)
- KernelBench: Can LLMs Write Efficient GPU Kernels? (ICML 2025 · similarity 0.85)
- Staggered Environment Resets Improve Massively Parallel On-Policy Reinforcement Learning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
