Research map / Architectures and efficiency / Mixture of experts
TEAM: Temporal–Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration
Linye Wei, Zixiang Luo, Pingzhi Tang, Meng Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add speculative exploration for hot tokens (SEH) (ablation) | vanilla SDAR 30B-A3B · HumanEval | Speedup / APT: 1.00 -> 1.43 speedup; APT 18.33 -> 12.23 | Improved |
| add delayed caching for decoded tokens (DCD) on top of SEH (ablation) | SDAR + SEH · HumanEval | Speedup / APT: 1.43 -> 2.01 speedup; APT 12.23 -> 7.79 | Improved |
| add limited activation for cold tokens (LAC) on top of SEH+DCD (ablation) | SDAR + SEH + DCD · HumanEval | Speedup / APT: 2.01 -> 2.20 speedup; APT 7.79 -> 6.80 | Improved |
| add LAC on top of SEH+DCD (ablation) | SDAR + SEH + DCD · MBPP | Speedup: 1.84 -> 2.08 | Improved |
| add full TEAM to SDAR | SDAR 30B-A3B · Average of 4 benchmarks | Score: 77.91 -> 77.84 | No clear effect |
| add full TEAM to SDAR | SDAR 30B-A3B · Average of 4 benchmarks | Speedup / activated experts per forward pass: 1x -> 1.94x; APF 54.99 -> 34.48 | Improved |
| never refresh the block cache (refresh-free) instead of refreshing every 4 steps (ablation) | DCD · Average of 4 benchmarks | Speedup: 1.29x -> 1.47x | Improved |
| never refresh the block cache instead of refreshing every 8 steps (ablation) | DCD · Average of 4 benchmarks | Score: 77.10 -> 77.57 | No clear effect |
| narrow hot-token set from (tau_h 0.7, L_h 3) to (tau_h 0.8, L_h 2) (ablation) | TEAM hot-token classification · HumanEval/MBPP average | Score: 73.10 -> 70.05 | Hurt |
| raise tau_h from 0.4 to 0.7 and lower L_h from 6 to 3 (ablation) | TEAM hot-token classification · HumanEval/MBPP average | APF: 23.35 -> 22.37 | Improved |
| construct branches from aligned top-1/top-2 candidates rather than also accepting the 3rd-confidence token (ablation) | SEH · acceptance probability of candidates | acceptance rate | Improved |
| cache decoded tokens at every iteration like dKV-Cache (for block diffusion) | dLLMs with block diffusion · SDAR | acceleration | No clear effect |
Closest papers by meaning
- dMoE: dLLMs with Learnable Block Experts (NeurIPS 2026 · similarity 0.91)
- Mixture Compressor for Mixture-of-Experts LLMs Gains More (ICLR 2025 · similarity 0.90)
- FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models (NeurIPS 2026 · similarity 0.89)
- MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance (ICML 2025 · similarity 0.89)
- MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems (NeurIPS 2025 · similarity 0.89)
- Pruning and Distilling Mixture-of-Experts into Dense Language Models (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
