AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Talon: Breaking the Synchronization Barrier in Speculative Decoding with Hybrid Model-based and Retrieve-based Drafting

Xiangxiang Gao, Weisheng Xie, , Lixin, Xuwei Fang, Chen Hang, Changqun Li, Yuhan Lin, Xiaolong Xu

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Talon, a speculative decoding framework that hybridizes model-based and retrieval-based drafting to remove the synchronization barrier between draft generation and verification.

Problem

Speculative decoding's draft-then-verify pipeline suffers from a mutual waiting problem, where draft generation and verification depend sequentially on each other, limiting LLM inference acceleration.

Approach

Combines model-based drafting with retrieval-based drafting in a hybrid scheme so that drafting and verification no longer block each other, breaking the synchronization barrier.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace EAGLE-3 with Talon (async hybrid drafting)Vicuna-13B · MT-Bench (greedy)speedup: 5.58x -> 5.84xImproved
replace EAGLE-3 with Talon (async hybrid drafting)Vicuna-13B · CNN/DM (greedy)speedup: 5.01x -> 4.67xHurt
replace EAGLE-3 with Talon (async hybrid drafting)LLaMA-3.1 8B · MT-Bench (greedy)speedup: 4.40x -> 4.99xImproved
replace EAGLE-3 with Talon (async hybrid drafting)LLaMA-3.1 8B · Alpaca (greedy)speedup: 4.82x -> 4.98xNo clear effect
replace EAGLE-3 with Talon (async hybrid drafting)DeepSeek-R1-Distill-LLaMA 8B · MT-Bench (greedy)speedup: 4.05x -> 4.93xImproved
replace EAGLE-3 with Talon (async hybrid drafting)Vicuna-13B · MT-Bench (greedy)average acceptance length: 6.65 -> 8.11Improved
replace EAGLE-3 with Talon (async hybrid drafting)DeepSeek-R1-Distill-LLaMA 8B · GSM8K (greedy)speedup: 5.01x -> 5.43xImproved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Experimental setup

Builds on: speculative decoding

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy