Research map / Large language models / Reasoning and chain of thought
ARC-Decode: Accelerated Decoding with Risk-Bounded Acceptance
Ying Li, Zhaode Wang, Zhiwen Chen, chengfei lv, Huan Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ARC-Decode, a speculative decoding variant that uses risk-bounded acceptance to relax the over-rejection of the classical lossless verification rule under sampling.
Problem
Speculative decoding speedups under sampling (T>0) lag behind greedy decoding because the classical lossless verification rule over-rejects draft tokens.
Approach
Replaces the classical lossless acceptance rule with a risk-bounded acceptance criterion that accepts more draft tokens under sampling while bounding the resulting risk; details beyond the abstract are not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add entropy-guided pruning to LTS-only ARC (ablation) | ARC-Decode (Llama-3.1-8B) · MT-Bench | speedup: 2.28× -> 2.61× | Improved |
| add entropy-guided pruning to LTS-only ARC (ablation) | ARC-Decode (Llama-3.1-8B) · Alpaca | speedup: 2.06× -> 2.28× | Improved |
| add entropy-guided pruning to LTS-only ARC (ablation) | ARC-Decode (Llama-3.1-8B) · HumanEval | speedup: 2.08× -> 2.13× | Mixed |
| add LTS-only relaxed acceptance to EAGLE-3 (ablation) | EAGLE-3 (Llama-3.1-8B) · Alpaca | speedup: 1.42× -> 2.06× | Improved |
| add LTS-only relaxed acceptance to EAGLE-3 (ablation) | EAGLE-3 (Llama-3.1-8B) · MT-Bench | speedup: 1.84× -> 2.28× | Improved |
| use entropy-guided pruning instead of depth-based pruning (ablation) | ARC-Decode (Qwen3-8B) · MT-Bench | accept length: 2.53 -> 2.70 | Improved |
| use entropy-guided pruning instead of depth-based pruning (ablation) | ARC-Decode (Qwen3-8B) · MT-Bench | tokens/s: 51.2 -> 56.4 | Improved |
| use entropy-guided pruning instead of cumulative-probability pruning (ablation) | ARC-Decode (Qwen3-8B) · MT-Bench | tokens/s: 51.8 -> 56.4 | Improved |
| raise sampling temperature from 0.1 to 0.9 (ablation) | ARC-Decode (Llama-3.1-8B) · MT-Bench | speedup: 3.13× -> 2.66× | Hurt |
| raise LTS threshold theta from 0.3 to 0.9 (ablation) | ARC-Decode (Llama-3.1-8B) · four tasks | speedup | Hurt |
| replace EAGLE-3 with ARC-Decode | Llama-3.1-8B · HumanEval | pass@10 (%): 69.9 -> 68.2 | No clear effect |
| replace EAGLE-3 with ARC-Decode | Llama-3.1-8B · MT-Bench | GPT-4o score: 6.77 -> 7.02 | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- EAGLE-3 — EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test (NeurIPS 2025) · better on 17 of 17 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Experimental setup
Builds on: speculative decoding
Closest papers by meaning
- Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment (ICLR 2025 · similarity 0.92)
- Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling (ICLR 2026 · similarity 0.92)
- LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding (ICML 2026 · similarity 0.91)
- Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match (ICLR 2026 · similarity 0.91)
- A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.90)
- Block Verification Accelerates Speculative Decoding (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
