Research map / Large language models / Agents, code and math
Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses an LLM-driven generative optimizer (Trace) with a domain-specific language and rich natural-language feedback (AutoGuide) to automatically generate and tune Legion parallel mappers, outperforming OpenTuner and matching or beating expert mappers.
Problem
Writing high-performance mappers for task-based parallel programs (Legion) requires days of expert manual tuning and exposes low-level system C++ APIs; existing autotuners like OpenTuner rely on scalar feedback and search inefficiently.
Approach
An Agent-System Interface combining a declarative mapper DSL (defining a structured search space and compiling to C++) with AutoGuide, a keyword-matching feedback mechanism that explains errors and suggests mapper edits, used inside a Trace-based generative optimizer with an LLM.
Key results
- Best Trace-found mappers match or exceed expert mappers across nine benchmarks, with up to 1.34x speedup on Circuit.
- Trace outperforms OpenTuner by 11x with 10 iterations each and still achieves 3.8x advantage over OpenTuner at 1000 iterations.
- DSL iterative refinement reaches 100% code generation success rate versus 0% for C++ in the same setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Trace generative optimization instead of OpenTuner for 10 iterations | LLM mapper optimization · 9 benchmarks | relative throughput: 11× | Helped |
| use Trace for 10 iterations instead of OpenTuner for 1000 iterations | LLM mapper optimization · 9 benchmarks | relative throughput: 3.8× | Helped |
| use DSL instead of C++ for single-trial code generation (ablation) | LLM mapper code generation · 10 mapping strategies | success rate: 0% -> 80% | Helped |
| use DSL instead of C++ for iterative-refinement code generation (ablation) | LLM mapper code generation · 10 mapping strategies | success rate: 0% -> 100% | Helped |
| add iterative refinement with compiler feedback (ablation) | DSL code generation · 10 mapping strategies | success rate: 80% -> 100% | Helped |
| add iterative refinement with compiler feedback (ablation) | C++ code generation · 10 mapping strategies | success rate: 0% -> 0% | No clear effect |
| use full Execution+Explain+Suggest feedback instead of reduced feedback variants (ablation) | Trace mapper optimization · 3 benchmarks | normalized throughput | Helped |
| add feedback-based iterative refinement instead of 0-shot generation (ablation) | LLM mapper optimization · 3 benchmarks | normalized throughput | Helped |
| add enriched explanations and mapper suggestions to raw execution feedback (ablation) | AutoGuide mapper optimization · 3 benchmarks | normalized throughput | Helped |
Surprising findings
- OpenTuner failed to produce valid mappers for 8 of 9 benchmarks in the reported runs, with mean throughput 0 for most.
- Despite having no pretraining examples, the DSL achieved higher LLM generation success than C++, which is widely represented in training corpora.
Limitations
- Evaluation limited to one node with four P100 GPUs and a single LLM (gpt-4o).
- Best-of-5-runs reporting; mean and worst-case throughput show occasional invalid mappers (0x) in SUMMA, PUMMA, Solomonik's.
- AutoGuide relies on keyword matching over raw output, which is hand-designed.
Experimental setup
Benchmarks: Circuit Stencil Pennant Cannon's SUMMA PUMMA Johnson's Solomonik's COSMA
Metrics: normalized throughput speedup code generation success rate lines of code
Baselines: Expert-written mappers Random mappers OPRO OpenTuner
Builds on: Trace OPRO OpenTuner DSPy least-to-most prompting
Closest papers by meaning
- Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows (NeurIPS 2026 · similarity 0.86)
- EvoCodeBench: Evaluating Coding Agents in Multi-Turn Iterative Interactions (NeurIPS 2026 · similarity 0.86)
- ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management (ICML 2026 · similarity 0.86)
- A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systems (ICML 2026 · similarity 0.85)
- CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control (ICML 2026 · similarity 0.85)
- LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.