Research map / Architectures and efficiency / RNNs and normalization
PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding
Yunhe Han, Yunqi Gao, Bing Hu, Mahdi Boloursaz Mashhadi, Yitong Duan, Pei Xiao, Yanfeng Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add token-batch pipeline scheduling (overlap draft generation and communication) (ablation) | PipeSD · HumanEval, Scenario 1 | time per accepted token (ms): 147 -> 129 | Improved |
| use dual-threshold NAV triggering instead of fixed-length triggering (ablation) | PipeSD · HumanEval, Scenario 1 | time per accepted token (ms): 164 -> 129 | Improved |
| use dual-threshold instead of token-level confidence only triggering (ablation) | PipeSD · HumanEval, Scenario 1 | time per accepted token (ms): 137 -> 129 | Improved |
| use dual-threshold instead of sequence-level confidence only triggering (ablation) | PipeSD · HumanEval, Scenario 1 | time per accepted token (ms): 139 -> 129 | Improved |
| tune thresholds with BO autotuner instead of grid search (ablation) | PipeSD · HumanEval, Scenario 1 | time per accepted token (ms): 139 -> 129 | Improved |
| tune thresholds with BO autotuner instead of random search (ablation) | PipeSD · GSM8K, Scenario 1 | time per accepted token (ms): 162 -> 145 | Improved |
| use BO autotuner instead of fixed (R1, R2) thresholds (ablation) | PipeSD · HumanEval, Scenario 1 | time per accepted token (ms): 138 (best fixed 0.9, 0.9) -> 129 | Improved |
| use PipeSD instead of Vanilla, HSL and EdgeLLM on a weak edge device (Scenario 3) | PipeSD · Scenario 3, GSM8K | speedup over Vanilla: 2.16x | Improved |
| use PipeSD versus baselines at high uplink bandwidth (80 Mbps) | PipeSD · HumanEval, Scenario 1, uplink 80 Mbps | speedup over Vanilla: 1.47x at 20 Mbps -> 1.34x at 80 Mbps | Mixed |
| use PipeSD instead of baselines for cloud energy | PipeSD · HumanEval, Scenario 1 | cloud energy reduction (%): 17.6%, 21.1%, 25.3% vs Vanilla, HSL, EdgeLLM | Improved |
| use dual-threshold triggering instead of HSL and EdgeLLM triggering | PipeSD · HumanEval, Scenario 1 | acceptance rate: 0.9148 (HSL) / 0.8917 (EdgeLLM) -> 0.9616 | Improved |
| add DP token batching versus heuristic pipelined policies (ablation) | PipeSD · HumanEval, Scenario 1 | time per accepted token | Improved |
Closest papers by meaning
- HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge-Cloud LLM Inference (ICML 2026 · similarity 0.90)
- E$^2$-SCI: Elastic Edge-Cloud Speculative Decoding via Credit Inertia (CVPR 2026 · similarity 0.89)
- SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMs (NeurIPS 2025 · similarity 0.86)
- SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration (ICLR 2025 · similarity 0.86)
- VIA-SD: Verification via Intra-Model Routing for Speculative Decoding (ICML 2026 · similarity 0.86)
- Yggdrasil: Bridging Dynamic Speculation and Static Runtime for Latency-Optimal Tree-Based LLM Decoding (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
