Research map / Large language models / Reasoning and chain of thought

E$^2$-SCI: Elastic Edge-Cloud Speculative Decoding via Credit Inertia

CVPR 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes E2-SCI, an edge-cloud speculative decoding framework that adapts draft-token verification thresholds using recent acceptance history ("credit inertia") and adds Progressive Lookahead Concurrency for asynchronous draft-verify pipelining.

Problem

Speculative decoding in edge-cloud settings is limited by uplink transmission latency, cloud verification cost, and rigid fixed acceptance thresholds that reject good draft tokens.

Approach

Observes that token acceptance rates across adjacent speculative windows are temporally correlated ("credit inertia") and uses an EWMA of recent acceptance to relax the divergence threshold for high-performing histories and tighten it for poor ones, with a learned predictor mapping state to threshold and window size; PLC overlaps draft generation with asynchronous cloud verification.

Key results

• E2-SCI reaches over 9.4 tokens/s on DeepSeek-R1-Distill-Qwen (1.5B/32B) and an 88.5% speed improvement over the FSD baseline, per the abstract.
• Table 1 reports speedups of 1.91x to 2.07x for E2-SCI across Qwen2, Llama3.1, Gemma2, and DeepSeek-R1 pairs, versus 1.29x to 2.02x for other methods.
• Integrated into Medusa and FSD, E2-SCI improves accuracy by +1.8-2.3% and +1.9-2.4% respectively across the four benchmarks (Table 3).

Limitations

• Threshold predictor is trained from offline calibration on small development subsets, and generalization to unseen workloads is only lightly validated.
• Evaluation simulates edge-cloud collaboration on a distributed testbed rather than a real deployment.
• Central claims depend on a credit inertia phenomenon shown mainly on MMLU and QuALITY, with limited statistical characterization.

Experimental setup

Datasets: GSM8K CommonsenseQA MMLU HumanEval

Benchmarks: GSM8K CommonsenseQA MMLU HumanEval

Metrics: speedup ratio tokens/s accuracy latency (s/token)

Baselines: SD FSD Medusa FR-Spec AdaDecode LR AMUSD PipeSpec EAGLE3

Builds on: Speculative decoding FSD Medusa EAGLE-3

Closest papers by meaning

• Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding (ICLR 2026 · similarity 0.87)
• Block Verification Accelerates Speculative Decoding (ICLR 2025 · similarity 0.86)
• PEARL: Parallel Speculative Decoding with Adaptive Draft Length (ICLR 2025 · similarity 0.86)
• Speculative Speculative Decoding (ICLR 2026 · similarity 0.86)
• Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing (NeurIPS 2026 · similarity 0.86)
• Traversal Verification for Speculative Tree Decoding (NeurIPS 2025 · similarity 0.85)