Research map / Federated learning and privacy / Communication-efficient FL

LBI-FL: Low-Bit Integerized Federated Learning with Temporally Dynamic Bit-Width Allocation

ICML 2025 · Federated learning and privacy › Communication-efficient FL · approach: Communication-efficient heterogeneous FL

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LBI-FL, a federated learning framework that quantizes weights, activations, and gradients below INT8 using a DQN agent that dynamically allocates INT4/INT6/INT8 bit-widths over training, reducing communication and BitOPs.

Problem

Federated learning is limited by uplink/downlink communication and client computation; existing quantization methods do not reduce both, and fixed-threshold bit-width schemes do not adapt across models and non-iid data.

Approach

A lightweight DQN agent, pretrained on 10% of CIFAR-10 with ResNet-20 and warm-started by behaviour cloning from teacher samples, chooses per-stage bit-width increases for weights, activations, and gradients using a reward combining a BitOPs-based quantization index and an accuracy index relative to INT8 training; the agent transfers to other architectures and non-iid FL.

Key results

• On CIFAR-10, LBI-FL reduces BitOPs by 51.5% (LeNet), 54.1% (ResNet-18), 47.7% (ResNet-50), 49.5% (ViT-S), 51.4% (MobileNetV2) relative to INT8 training, with accuracy loss of about 1.5% or less in most cases (ResNet-18: 84.16 vs 84.23 for UI8).
• Communication cost is reduced by 87.5% vs FP32 and 50% vs INT8 because weights stay at INT4.
• Under non-iid Dirichlet 0.25 on CIFAR-10, LBI-FL reaches 78.56% vs 78.71% for UI8 at 52.03% BitOPs reduction.

Measured effects reported in the paper

Surprising findings

• Switching weights from INT4 to INT8 or INT8 to INT4 mid-training shows that lowering bit-width late in training can cause collapse (NaN for MobileNetV2), motivating temporal allocation.
• Random selection of INT4 clients performs acceptably on LeNet but loses 1-2 points on ResNet-18 and MobileNetV2.

Limitations

• Only weights are kept at INT4 in practice; the agent learned to fix weights at INT4, so the mixed-precision search is effectively over activations and gradients.
• Convergence analysis covers only gradient quantization and assumes zero-mean quantization noise; weight and activation effects are not analyzed.
• Experiments are limited to small image classification benchmarks (CIFAR-10/100) with simulated FL setups of 10 to 100 clients.

Experimental setup

Datasets: CIFAR-10 CIFAR-100

Benchmarks: CIFAR-10 CIFAR-100

Metrics: Top-1 accuracy BitOPs reduction ratio communication cost reduction epochs to target accuracy

Baselines: FP32 UI4 UI8 Random bit-width selection

Builds on: FedAvg DQN Zhu et al. 2020 directional sensitive gradient clipping (INT8 training) Zhang et al. 2020 quantization loss QSGD

Closest papers by meaning

• Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity (AAAI 2024 · similarity 0.89)
• Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices (CVPR 2024 · similarity 0.89)
• Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.87)
• Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis (NeurIPS 2024 · similarity 0.86)
• Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainable (NeurIPS 2025 · similarity 0.86)
• Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.85)