Research map / Large language models / Reasoning and chain of thought

Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Cactus, a training-free speculative sampling acceptance rule derived from constrained optimization that bounds KL divergence from the verifier while raising acceptance rates over SpS and TAS.

Problem

Speculative sampling enforces exact verifier distribution matching, rejecting correct but lower-probability tokens, while typical acceptance sampling raises acceptance by heuristics that distort the verifier distribution and can degrade quality.

Approach

Reformulates speculative sampling as constrained optimization over a target distribution h that maximizes drafted-token acceptance subject to a KL-divergence budget from the verifier; the closed form (second-order Taylor approximation) boosts the drafted token's probability by a small bonus sqrt(2δq(1-q)) with only element-wise operations.

Key results

• On GSM8K with Qwen3-14B verifier and 0.6B drafter (m=20), Cactus 1.0 reaches AL 7.00-7.61 range while keeping accuracy near verifier (e.g., 92.87 with 14B), versus SpS AL 5.11.
• On GPQA with Qwen3-8B verifier (m=20), TAS drops accuracy to 38.89 vs SpS 42.93, while Cactus 0.75 keeps 40.01 with AL 6.89.
• On Spec-Bench, Cactus (delta=1) achieves overall 1.88x speedup versus 1.81x for SpS with Qwen3-14B verifier.

Surprising findings

• TAS and Cactus both raise acceptance but TAS reduces GPQA accuracy notably below SpS, whereas Cactus does not degrade it.
• Mentored decoding and SpecCas degrade IFEval/GSM8K quality substantially in some settings despite higher acceptance.

Limitations

• Taylor approximation of the KL constraint is justified heuristically for small delta and may be inexact for larger delta.
• Speedups measured in HuggingFace Transformers for Spec-Bench; wall-time results on vLLM are limited to some settings.
• Experiments restricted to same-family drafter-verifier pairs and sampling with top-k/top-p/temperature; multi-drafter or tree extensions left to future work.

Experimental setup

Datasets: GSM8K IFEval GPQA Spec-Bench

Benchmarks: GSM8K IFEval GPQA Spec-Bench

Metrics: strict-match accuracy acceptance length (AL) rejected tokens (Rej) wall-time speedup

Baselines: SpS TAS Top-k decoding Mentored decoding SpecCas Interpolation

Builds on: speculative sampling typical acceptance sampling Medusa EAGLE Judge Decoding

Closest papers by meaning

• LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding (ICML 2026 · similarity 0.91)
• A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.91)
• Global Resolution: Optimal Multi-Draft Speculative Sampling via Convex Optimization (ICLR 2026 · similarity 0.89)
• Towards Optimal Multi-draft Speculative Decoding (ICLR 2025 · similarity 0.89)
• SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.89)
• Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment (ICLR 2025 · similarity 0.89)