Research map / Large language models / Agents, code and math
SOLAR: AI-Powered Speed-of-Light Performance Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SOLAR, a framework that derives validated speed-of-light (roofline) performance bounds directly from PyTorch and JAX source code via an LLM-translated, numerically validated Affine Loop IR and a deterministic einsum-graph analytical backend.
Problem
Speed-of-Light bounds (theoretical minimum execution time of a DL workload on given hardware) are manual, error-prone to derive, and disconnected from model development; existing FLOP counters, profilers, and LLM estimates do not provide validated, graph-level, cache-aware bounds.
Approach
Confine the LLM to a verifiable task (translating source code into an executable Affine Loop IR checked by output comparison), then deterministically lift the IR to an einsum graph and compute unfused, fused, and cache-aware roofline bounds, plus inverse-roofline hardware provisioning.
Key results
- Fused SOL analysis reveals up to 7.8x additional headroom beyond per-operator analysis on KernelBench; geomean headroom is 4.3x (L1), 21.6x (L2), 54.6x (L3), 10.2x (L4) in eager mode.
- Cache-aware bounds tighten SOL by 2.06x (up to 2.25x per the evaluation); optimal DeepSeek MLA chain order gives 2.04x FLOP reduction over naive left-to-right.
- Inverse roofline shows pi0 needs 6.4x current bandwidth for 500 Hz on Jetson Thor, and DreamZero WAM needs 19.7x bandwidth and 8.3x compute.
Surprising findings
- Zero-shot LLM SOL estimation achieves 83% on KernelBench overall but only 38% on L3 subgraphs.
- Cache-aware SOL is tighter than unfused on L1 but looser on L2, so the most informative fidelity level depends on the workload.
Limitations
- Analysis depends only on tensor shapes, not values, so value-dependent optimizations like compression or constant propagation are not captured.
- Translation validation by sampled inputs gives no formal guarantee; incorrect translations may pass.
- SOL bound may not be tight in practice due to power capping and thermal throttling.
Experimental setup
Datasets: KernelBench JAX/Flax models SOL-ExecBench
Benchmarks: KernelBench SOL-ExecBench JAX/Flax models robotics workloads
Metrics: SOL speedup (runtime / SOL bound) validation coverage SOL violations FLOP reduction required hardware improvement
Baselines: fvcore ptflops torchinfo Pure LLM SOL (Claude Code with Opus 4.6) PyTorch eager torch.compile NCU NSys
Builds on: roofline model Orojenesis Timeloop AccelForge KernelBench
Closest papers by meaning
- SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation (NeurIPS 2024 · similarity 0.82)
- The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization (NeurIPS 2025 · similarity 0.82)
- Integrated Hardware Architecture and Device Placement Search (ICML 2024 · similarity 0.81)
- The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvements (NeurIPS 2025 · similarity 0.81)
- TinyTTA: Efficient Test-time Adaptation via Early-exit Ensembles on Edge Devices (NeurIPS 2024 · similarity 0.81)
- Kinetics: Rethinking Test-Time Scaling Law (NeurIPS 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.