Research map / Reinforcement learning / RL from feedback
Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch
Ziyang Zhang, Xinheng Ding, Jiayi Yuan, Rixin Liu, Huizi Mao, Jiarong Xing, Zirui Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Tree-Based Invariant Kernels on top of BIO (ablation) | vLLM Qwen3-8B · AIME24 across TP and BS | count of unique outputs: 7.87 -> 1 | Improved |
| add TBIK on top of BIO (ablation) | vLLM Llama-3.1-8B-Instruct · AMC23 | max probability divergence (x1e-3): 21.06 -> 0 | Improved |
| add batch-invariant operations only (ablation) | vLLM BF16 · AIME24/AMC23 across TP and BS | count of unique outputs: 12.00 -> 7.87 (Qwen3-8B AIME24) | Mixed |
| add BIO only (ablation) | vLLM Llama-3.1-8B-Instruct · AIME24 | max probability divergence (x1e-3): 26.48 -> 27.54 | Hurt |
| add TBIK to vLLM and FSDP in RL | GRPO Qwen3-1.7B · GSM8K | Pass@1: 0.60 -> 0.73 | Improved |
| add TBIK to vLLM and FSDP in RL | GRPO Qwen3-1.7B · GSM8K | KL between rollout and training engines | Improved |
| add BIO only in RL (ablation) | GRPO Qwen3-1.7B · GSM8K | Pass@1: 0.60 -> 0.68 | Improved |
| add TBIK on top of BIO (ablation) | SGLang Qwen3-8B · latency in/out 4096/4096 | end-to-end latency (s): 125.64 -> 130.36 | Hurt |
| use tree-based MatMul kernel instead of cuBLAS at large M (ablation) | Tree-Based MatMul · BF16 kernel throughput | TFLOPS: 190 -> 120 TFLOPS | Hurt |
| use FP32 tree-based MatMul instead of cuBLAS (ablation) | Tree-Based MatMul · FP32 kernel throughput | TFLOPS | No clear effect |
Closest papers by meaning
- Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference (NeurIPS 2025 · similarity 0.88)
- SPD: Sync-Point Drop for Efficient Tensor Parallelism of Large Language Models (ICML 2025 · similarity 0.88)
- Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity (NeurIPS 2025 · similarity 0.88)
- Synergistic Tensor and Pipeline Parallelism (NeurIPS 2025 · similarity 0.87)
- Generalized Parallel Scaling with Interdependent Generations (ICLR 2026 · similarity 0.87)
- Progressive Mixed-Precision Decoding for Efficient LLM Inference (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
