Research map / Large language models / Reasoning and chain of thought

ConFu: Contemplate the Future for Better Speculative Sampling

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ConFu, a speculative decoding draft head that conditions on learnable contemplate tokens (MoE-parameterized) and soft prompts from the frozen target model to anticipate future generation direction, trained with anchor token sampling and future prediction replication.

Problem

Draft models in speculative decoding, including EAGLE-series heads, condition only on the current prefix, so their predictions drift from the target model over decoding steps and acceptance rates decline.

Approach

Append a future-prediction token, produced from target-model contemplate tokens with learnable soft prompts and a mixture-of-experts parameterization conditioned on the last accepted token, to the EAGLE-3 draft head; train with anchor-token sampling and by replicating future predictions onto nearby non-anchor tokens.

Key results

• On Llama-3.2-3B-Instruct at temperature 0 with 30 draft nodes, ConFu reaches average tau 4.41 and speed-up 2.11 vs EAGLE-3's 4.00 and 1.83.
• On Llama-3.1-8B-Instruct at temperature 0 with 30 draft nodes, ConFu reaches average tau 5.01 and speed-up 2.69 vs EAGLE-3's 4.59 and 2.36.
• On Qwen3-4B, ConFu improves average tau by 21.0% and speed-up ratio by 19.8% over EAGLE-3 on average.

Surprising findings

• Continuing to train the EAGLE-3 baseline for the same extra steps gave no measurable improvement, per the authors.

Limitations

• Inserting one contemplate token per draft node adds computation proportional to the number of draft nodes.
• The 3B EAGLE-3 baseline was trained by the authors, while the 8B baseline uses an official checkpoint.
• Evaluation is limited to SpecBench with batch size 1 on a single H100 GPU.

Experimental setup

Datasets: ShareGPT UltraChat-200K SpecBench

Benchmarks: SpecBench

Metrics: average accepted length (tau) speed-up ratio

Baselines: EAGLE-3 Medusa HASS

Builds on: EAGLE-3 BiTA pause tokens EAGLE

Closest papers by meaning

• Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment (ICLR 2025 · similarity 0.91)
• A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.90)
• TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees (NeurIPS 2026 · similarity 0.90)
• SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.90)
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios (AAAI 2026 · similarity 0.90)
• Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies (ICML 2025 · similarity 0.90)