AI Paper Atlas

Research map / Federated learning and privacy / Personalized and heterogeneous FL

Efficient Collaborative LLM Fine-Tuning over Heterogeneous Mobile Devices via Many Backbones to One Side-Network Tuning

Xingke Yang, Liang Li, Sicong Li, Liwei Guan, Hao Wang, Xiaoqi Qin, Jiang Liu, Xin Fu, Miao Pan

NeurIPS 2026 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Federated prompt and LoRA tuning · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a system for collaborative LLM fine-tuning across heterogeneous mobile devices, addressing on-device resource overhead, straggler-prone synchronous aggregation, and the single-shared-backbone assumption.

Problem

Federated LLM fine-tuning over heterogeneous mobile devices faces prohibitive on-device resource overhead, straggler-prone synchronous aggregation, and the assumption of a unified pretrained backbone.

Approach

Uses many backbones mapped to one side-network tuning scheme for collaborative fine-tuning on heterogeneous mobile devices; details beyond the abstract are not available in the provided text.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add token selection (upload only loss-relevant activations) (ablation)EC-MobiLLM · MRPC heterogeneouscommunication / TTA: 1.1 MB -> 281.6 MB without; TTA 5.9 -> 45.7 min withoutImproved
add server caching of activations (ablation)EC-MobiLLM · MRPC heterogeneouscomputation / TTA: 2.7 -> 54.0 TFLOPs without; TTA 5.9 -> 91.3 min withoutImproved
add non-blocking asynchronous collaboration (ablation)EC-MobiLLM · MRPC heterogeneousTTA (min): 9.6 -> 5.9Improved
ablate TS, SC or NBAC (ablation)EC-MobiLLM · MRPC heterogeneousaccuracy: 88.1 in all variantsNo clear effect
train shared side-network collaboratively (Global) instead of per-device (Single) (ablation)EC-MobiLLM · SST-2accuracy: TX2 88.4 -> 92.1; AGX 90.0 -> 92.9Improved
allow device-specific larger backbones instead of shared small backbone with LoRA rank tuningEC-MobiLLM · SST-2 AGXaccuracy: FLoRA 90.8 -> 92.9Improved
use average-interval layer sampling instead of importance-based selection (ablation)EC-MobiLLM · SST-2 alignment studyaccuracyNo clear effect
use more backbone activation layers (ablation)EC-MobiLLM · OPT-1.3B/RoBERTa-LargeaccuracyImproved
force side-network hidden size to mismatch backbone (e.g. 2048 for OPT-125M) (ablation)EC-MobiLLM · SST-2accuracyHurt
upload privatized activations once and cache vs repeated releases under same DP budget (ablation)EC-MobiLLM · SST-2 DPaccuracy: AGX 86.5 -> 89.2Improved
add DP noise to activations (one-shot)EC-MobiLLM · SST-2 AGXaccuracy: 92.9 -> 89.2Hurt
replace LoRA/BitFit federated training with server-side side-network tuningEC-MobiLLM · MRPC centralizedaccuracy: 91.5 (LoRA) -> 89.6Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy