Research map / Federated learning and privacy / Privacy attacks and secure learning
FedSEA-LLaMA: A Secure, Efficient and Adaptive Federated Splitting Framework for Large Language Models
Zishuai Zhang, Hainan Zhang, Weihua Li, Qinnan Zhang, Jin Dong, Yongxin Tong, Zhiming Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add collaborative KV cache (ablation) | FedSEA-LLaMA · Query length 4000 inference | inference speed: 8.2 times that without it | Improved |
| add attention-mask compression (send seq_len, pad_len) (ablation) | FedSEA-LLaMA · Distributed forward pass | time per token: average time reduction of 87.6% | Improved |
| use FedSEA-LLaMA instead of FL-GLM for distributed inference | FL-GLM · Distributed inference | speedup: over 7× speedup | Improved |
| split model so client holds first/last blocks instead of full model | LLaMA2-7B · ReCoRD client GPU memory | GPU memory: 28.2G -> 3.4G | Improved |
| increase noise scale from 0.0 to 0.1 (ablation) | FedSEA-LLaMA Blocks_A=1 · MultiRC | F1a/EM: 77.48/40.82 -> 59.11/8.18 | Hurt |
| increase noise scale from 0.0 to 0.1 (ablation) | FedSEA-LLaMA Blocks_A=3 · MultiRC | F1a/EM: 77.89/41.34 -> 75.68/34.10 | Hurt |
| increase Blocks_A from 1 to 3 at noise 0.1 (ablation) | FedSEA-LLaMA · MultiRC | F1a/EM: 59.11/8.18 -> 75.68/34.10 | Improved |
| increase noise scale from 0.0 to 0.02 (ablation) | FedSEA-LLaMA Blocks_A=3 · MultiRC | F1a/EM: 77.89/41.34 -> 78.61/42.39 | No clear effect |
| vary Blocks_A and Blocks_C from 1 to 3 (no noise) (ablation) | FedSEA-LLaMA · MultiRC | F1a/EM: 80.34/44.60 -> 81.17/47.22 | No clear effect |
| keep embedding plus client blocks instead of embedding only (ablation) | FedSEA-LLaMA · ReCoRD model inversion | BLEU-4 / Rouge-2: 0.051/0.003 vs FedBert 7.053 and 28.57 | Improved |
| add Gaussian noise 0.02 to hidden states (Blocks_A=3) (ablation) | FedSEA-LLaMA · ReCoRD model inversion | Rouge-2 F1/BLEU-4: 0.069/0.016 -> 0.084/0.044 | No clear effect |
Closest papers by meaning
- Differentially Private Federated Low Rank Adaptation Beyond Fixed-Matrix (NeurIPS 2025 · similarity 0.89)
- Split-and-Denoise: Protect large language model inference with local differential privacy (ICML 2024 · similarity 0.89)
- FedRW: Efficient Privacy-Preserving Data Reweighting for Enhancing Federated Learning of Language Models (NeurIPS 2025 · similarity 0.88)
- Splitting with Importance-aware Updating for Heterogeneous Federated Learning with Large Language Models (ICML 2025 · similarity 0.87)
- FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models (NeurIPS 2025 · similarity 0.87)
- FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
