Research map / Large language models / Language, safety and interpretability
Fast and Expressive Multi-Byte Prediction with Probabilistic Circuits
Andreas Grivas, Lorenzo Loconte, Emile van Krieken, Piotr Nawrot, Yu Zhao, Euan Wielewski, Pasquale Minervini, Edoardo Ponti, Antonio Vergari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase CP mixture components r from 1 (FF) to 128 (ablation) | MTPC-CP · EvaByte, speculative sampling, n=8 | mean accepted tokens: 5.19 -> 6.02 | Improved |
| increase CP mixture components r from 32 to 128 (ablation) | MTPC-CP · EvaByte, speculative sampling, n=8 | throughput (tok/s): 191.0 -> 182.3 | Hurt |
| use CP mixture (r=32) instead of fully factorised heads (ablation) | EvaByte-MTP · EvaByte, speculative sampling, n=8 | throughput (tok/s): 176.3 -> 191.0 | Improved |
| use HMM circuit head instead of BTree (ablation) | MTPC · EvaByte and Llama, n=8 and n=16 | throughput | Hurt |
| extend MTP window from n=8 to n=16 with BTree head (ablation) | MTPC-BTree · EvaByte and Llama, speculative sampling | throughput | Mixed |
| use expressive circuits instead of factorised head (ablation) | EvaByte-MTP · EvaByte, greedy speculative decoding | throughput | Hurt |
| add LoRA layers to the draft backbone (n=16) (ablation) | MTPC-BTree · EvaByte and Llama, speculative sampling and greedy | mean accepted tokens: +.69 with 4 LoRA layers (EvaByte, sampling) | Improved |
| use 1 LoRA layer rather than 4 for EvaByte (ablation) | MTPC-BTree · EvaByte, n=16 | throughput | Improved |
| add LoRA layers to EvaByte-MTP with n=8 (ablation) | EvaByte-MTP (FF) · EvaByte, n=8 | mean accepted tokens | No clear effect |
| train LoRA adapters on last 16 of 32 layers of EvaByte (ablation) | EvaByte draft · EvaByte | mean accepted tokens vs latency: +37% accepted, latency 1.5x forward pass | Mixed |
| train more than 16 layers with LoRA (ablation) | EvaByte draft · EvaByte | acceptance rate | No clear effect |
| replace fully factorised MTP with BTree PC head plus LoRA, within speculative decoding | EvaByte-MTP · Tulu3 validation, speculative sampling | speedup: 1.17x over best FF MTP | Improved |
Closest papers by meaning
- Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing (ICML 2026 · similarity 0.90)
- Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding (ICML 2025 · similarity 0.89)
- MARS: Enabling Autoregressive Models Multi-Token Generation (NeurIPS 2026 · similarity 0.88)
- Better & Faster Large Language Models via Multi-token Prediction (ICML 2024 · similarity 0.88)
- AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism (ICML 2025 · similarity 0.88)
- Optimized Multi-Token Joint Decoding With Auxiliary Model for LLM Inference (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
