AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

RealtimeTool: Parallel Decoding for Real-Time LLM Function Calling

Xiaoxin Shi, Jiaxin Wan, Linkang Dong, Wei Jiang, Yue Liu, Zengfeng Huang

ICML 2026 · Large language models › Language, safety and interpretability · approach: Parallel and early-exit decoding

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train SimpleTool special-token parallel decoding instead of baseline JSON outputQwen2.5/Qwen3 models 0.5B-14B · Five benchmarks averageoverall accuracy: +2.9% average; 70.9 -> 75.3 (0.5B)Improved
train SimpleTool special-token parallel decodingQwen models · Five benchmarks averagefunction accuracy: +7.1%Improved
train SimpleTool on Mobile ActionsQwen3-4B · Mobile Actionsoverall accuracy: 68.0 -> 84.5Improved
train SimpleTool on BFCL LiveQwen2.5-1.5B · BFCL Liveoverall accuracy: 67.4 -> 63.6Hurt
train SimpleTool on BFCL ExecQwen3-4B · BFCL Execoverall accuracy: 94.6 -> 89.9Hurt
compress low-entropy tokens with special tokens and decode heads in parallelQwen models · BFCL-v3 single-turn, vLLM, RTX 4090end-to-end speedup: 3.07x to 3.88xImproved
use parallel decoding with 8 heads vs single sequence (ablation)SimpleTool · Batch scaling across six modelsper-token overhead: +8.2% overhead, 93.0% efficiencyNo clear effect
increase LoRA rank from 64 to 1024 (ablation)ST-Qwen3-4B · BFCL-v3accuracy: 85.5 -> 87.0Improved
LoRA rank 512 vs 256 (ablation)ST-Qwen3-4B · BFCL-v3accuracy: 86.4 -> 85.3Hurt
add synthetic augmentation data to xLAM (ablation)ST-Qwen3-4B · BFCL-v3accuracy: 85.3 -> 86.3Improved
ST-Qwen-0.5B vs FunctionGemma after fine-tuningedge function calling · Mobile Actions fine-tunedaccuracy: 85.0 -> 86.2Improved
ST-Qwen-0.5B vs FunctionGemma latencyedge function calling · Mobile Actions latencyP90 latency (ms): 139.5 -> 74.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy