Research map / Architectures and efficiency / Mixture of experts
AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization
Qiyang Li, Rui Kong, Yuchen Li, Hengyi Cai, Shuaiqiang Wang, Linghe Kong, Guihai Chen, Dawei Yin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use token-level pre-gating with fused SGMM adapter switching | dynamic LoRA adapters, Llama2-7B · ShareGPT decoding | latency (ms/token): 8.5 (+254%) -> 3.1 (+29%) | Improved |
| use SGMM fused kernel instead of simple merge (ablation) | AdaFuse · ShareGPT decoding | latency (ms/token): 4.2 -> 3.1 | Improved |
| use simple layer-by-layer adapter merging | MoRAL · ShareGPT decoding | latency (ms/token): 8.5 -> 4.5 | Improved |
| switch from MoRAL layer-wise routing to AdaFuse pre-gating (with simple merge) (ablation) | dynamic adapters · ShareGPT decoding | latency (ms/token): 4.5 -> 4.2 | Improved |
| use AdaFuse token-level pre-gating instead of block-wise PESC | Llama2-7B · General ability average | accuracy average: 60.45 -> 60.12 | No clear effect |
| use AdaFuse instead of static LoRA | Llama2-7B · General ability average | accuracy average: 59.64 -> 60.12 | Improved |
| use AdaFuse instead of LoRA | Llama2-7B · CommonsenseQA | accuracy: 75.51 -> 79.03 | Improved |
| use AdaFuse instead of MoLA | Llama2-7B · ScienceQA/CommonsenseQA/OpenbookQA avg | accuracy average: 84.20 -> 83.60 | Hurt |
| use AdaFuse instead of LoRA | Mistral-7B · ScienceQA/CommonsenseQA/OpenbookQA avg | accuracy average: 86.06 -> 87.24 | Improved |
| use AdaFuse instead of MoRAL for memory | Llama2-7B · ShareGPT peak GPU memory | memory (GiB): 13.3 (+3%) -> 13.8 (+7%) | No clear effect |
Closest papers by meaning
- Ada-K Routing: Boosting the Efficiency of MoE-based LLMs (ICLR 2025 · similarity 0.88)
- $\textit{Read-ME}$: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design (NeurIPS 2024 · similarity 0.88)
- Oracle-MoE: Locality-preserving Routing in the Oracle Space for Memory-constrained Large Language Model Inference (ICML 2025 · similarity 0.87)
- AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism (ICML 2025 · similarity 0.87)
- Dr.LLM: Dynamic Layer Routing in LLMs (ICLR 2026 · similarity 0.87)
- Adaptive Draft-Verification for Efficient Large Language Model Decoding (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
