AI Paper Atlas

Research map / Large language models / Agents, code and math

DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programs

Size Zheng, Xuegui Zheng, Hanshi Sun, Qi Hou, Wenlei Bao, Shiyu Li, Haojie Duanmu, Jin Fang, Chenli Xue, Chenhui Huang, Yuanqiang Liu, Renze Chen and 7 more

ICML 2026 · Large language models › Agents, code and math · approach: LLM-guided program synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add cross-rank swizzle (ablation)DITRON GEMM+AllReduce · Llama 8xH800tokens/s: 11,605 -> 12,174Improved
add multi-level tiling (ablation)DITRON GEMM+AllReduce · Llama 8xH800tokens/s: 11,706 -> 12,174Improved
add computation-communication overlap (ablation)DITRON GEMM+AllReduce · Llama 8xH800tokens/s: 10,320 -> 12,174Improved
add cross-rank swizzle (ablation)DITRON AG+GEMM / GEMM+RS · Llama 8xH800tokens/s: 10,642 -> 10,928Improved
add multi-level tiling (ablation)DITRON AG+GEMM / GEMM+RS · Llama 8xH800tokens/s: 9,494 -> 10,928Improved
add overlap with AG+RS (ablation)DITRON · Llama 8xH800tokens/s: 10,320 -> 10,928Improved
use DITRON in vLLM at batch size below 128vLLM · Qwen3-32B/LLaMA3-70BthroughputHurt
use DITRON in vLLM at batch size above 128vLLM · Qwen3-32B/LLaMA3-70Bthroughput: 5%-30% speedupImproved
use AllReduce instead of AG+RS for attention moduleDITRON · Qwen3-32B attentionspeedupImproved
use AG+RS instead of AllReduce for FFN beyond 2k tokensDITRON · Qwen3-32B FFN, 128k tokensspeedup: 1.17x -> 1.27xImproved
scale TP workloads across nodes (8-32 GPUs)DITRON AG-GEMM/GEMM-RS · strong scalingspeedup over CuBLAS+NCCL: 0.80x to 1.71xMixed
distributed MegaKernel for single-batch inferenceDITRON · Qwen3/LLaMA3 8xH800latency per token: 6.28x vs Torch EagerImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy