AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

SparseInfer: Accelerating Large Language Model Inference with Semantics-Inspired Adaptive Sparse Activation

Qinsi Wang, Saeed Vahidian, Hancheng Ye, Jianyang Gu, Jianyi Zhang, Yiran Chen

ICML 2026 · Architectures and efficiency › RNNs and normalization · approach: Neural scaling laws

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use similarity-guided instead of stability-guided prediction on zero-shot TruthfulQA (ablation)CoreInfer (OPT-6.7b) · TruthfulQA zero-shotBLEU max: 6.22 -> 9.12Improved
use similarity-guided instead of stability-guided prediction on zero-shot TriviaQA (ablation)CoreInfer (OPT-6.7b) · TriviaQA zero-shotexact match: 12.0 -> 21.8Improved
use similarity-guided instead of stability-guided prediction on zero-shot wmt16-de-en (ablation)CoreInfer (OPT-6.7b) · wmt16-de-en zero-shotBLEU: 12.2 -> 26.3Improved
use similarity-guided instead of stability-guided prediction on zero-shot wmt16-ro-en (ablation)CoreInfer (OPT-6.7b) · wmt16-ro-en zero-shotBLEU: 3.36 -> 27.8Improved
use stability-guided prediction on few-shot and information extraction tasks (ablation)CoreInfer (OPT-6.7b) · Xsum / SQuAD / few-shot QA and translationtask metrics: 6.7 -> 6.3 Xsum rouge; 52.1 -> 53.2 SQuADNo clear effect
apply CoreInfer sentence-level sparse inferenceOPT-6.7b · Xsumrouge: 6.7 -> 6.3No clear effect
apply CoreInfer to LLaMA3.1-8bLLaMA3.1-8b · SQuADcontains: 54.3 -> 49.7Hurt
apply CoreInfer to OPT-13bOPT-13b · wmt16-de-en zero-shotBLEU: 31.3 -> 35.2Improved
use CoreInfer instead of PowerInfer on TITAN XPOPT-6.7b · decode speedtokens/s: 7.32 -> 19.83Improved
use CoreInfer instead of Huggingface Transformer on TITAN XPOPT-6.7b · speedupspeedup: 1x -> 10.33xImproved
increase core neuron ratio for translation (beta = gamma 0.2 to 0.4) (ablation)CoreInfer · Translation tasksBLEU: about 40% of neuronsImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy