Research map / Continual learning / Online and replay-based
Towards Efficient Replay in Federated Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Re-Fed, a federated incremental learning framework where each client caches replay samples ranked by gradient norms from a personalized informative model that blends local and global information.
Problem
Catastrophic forgetting under data heterogeneity (non-IID) in federated incremental learning, where edge clients have limited storage and cannot keep all past samples for replay.
Approach
Each client trains a personalized informative model (PIM) regularized toward the global model via a ratio factor, records per-sample gradient norms during PIM training, and caches the highest-importance previous samples for replay, so caching reflects both local and global data distribution without generating or reconstructing data.
Key results
- Re-Fed achieves best final accuracy in all six settings, with margins of 1.87% to 19.73% over baselines (e.g., CIFAR100 class-incremental 25.61% vs 23.02% for FCIL).
- On Digit10 with memory 2000, Re-Fed reaches 85.96% vs 77.59% for FedAvg.
- Re-Fed requires the fewest communication rounds to reach the reported accuracy on all datasets (e.g., CIFAR10 562 vs 574 for Shared).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Re-Fed sample caching and replay instead of FedCIL | Federated class-incremental learning · CIFAR10 (α=1.0) | accuracy: 27.35±1.24 -> 29.22±0.49 | Helped |
| Use Re-Fed sample caching and replay instead of FCIL | Federated class-incremental learning · CIFAR100 (α=5.0) | accuracy: 23.02±0.66 -> 25.61±0.88 | Helped |
| Use Re-Fed sample caching and replay instead of FCIL | Federated class-incremental learning · Tiny-ImageNet (α=10) | accuracy: 29.58±0.15 -> 32.07±0.27 | Helped |
| Use Re-Fed sample caching and replay instead of FedCIL | Federated domain-incremental learning · Digit10 (α=0.1) | accuracy: 83.85±0.80 -> 85.96±0.14 | Helped |
| Use Re-Fed sample caching and replay instead of FedCIL | Federated domain-incremental learning · Office31 (α=1) | accuracy: 46.26±2.24 -> 50.80±0.77 | Helped |
| Use Re-Fed sample caching and replay instead of FedAvg | Federated domain-incremental learning · DomainNet (α=10) | accuracy: 51.73±2.32 -> 56.66±0.50 | Helped |
| Use Re-Fed caching to reach the reported accuracy | Federated incremental learning · CIFAR10 (Task:5) | communication rounds to best test accuracy: 142±0.67 -> 562±1.67 | Hurt |
| Use Re-Fed caching to reach the reported accuracy | Federated incremental learning · CIFAR100 (Task:10) | communication rounds to best test accuracy: 137±2.00 -> 1039±4.33 | Hurt |
| Use Re-Fed caching to reach the reported accuracy | Federated incremental learning · Tiny-ImageNet (Task:10) | communication rounds to best test accuracy: 132±3.00 -> 1128±3.67 | Hurt |
| Use Re-Fed caching to reach the reported accuracy | Federated incremental learning · Digit10 (Task:4) | communication rounds to best test accuracy: 112±0.33 -> 325±1.33 | Hurt |
| Use Re-Fed caching to reach the reported accuracy | Federated incremental learning · Office31 (Task:3) | communication rounds to best test accuracy: 144±0.67 -> 388±1.67 | Hurt |
| Use Re-Fed caching to reach the reported accuracy | Federated incremental learning · DomainNet (Task:6) | communication rounds to best test accuracy: 141±1.67 -> 661±2.33 | Hurt |
Surprising findings
- Fixed-model baseline degrades strongly, and larger heterogeneity (smaller alpha) benefits from more reliance on global information in PIM, with optimal lambda decreasing as heterogeneity increases.
Limitations
- Does not consider dynamic edge-client requirements such as heterogeneous storage, computation, or task arrival times (stated as future work).
- Hyperparameter lambda must be tuned according to data heterogeneity level.
- Experiments limited to image classification with a ResNet18 backbone.
Experimental setup
Datasets: CIFAR10 CIFAR100 Tiny-ImageNet Digit10 Office31 DomainNet
Benchmarks: CIFAR10 CIFAR100 Tiny-ImageNet Digit10 Office31 DomainNet
Metrics: top-1 accuracy top-10 accuracy communication rounds to best accuracy
Baselines: FedAvg FedProx Fixed DANN+FL Shared FCIL FedCIL
Builds on: FedAvg FedProx FedCIL DANN
Closest papers by meaning
- Accurate Forgetting for Heterogeneous Federated Continual Learning (ICLR 2024 · similarity 0.91)
- CAN: Leveraging Clients As Navigators for Generative Replay in Federated Continual Learning (ICML 2025 · similarity 0.89)
- Class-wise Balancing Data Replay for Federated Class-Incremental Learning (NeurIPS 2025 · similarity 0.88)
- Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting (ICLR 2025 · similarity 0.88)
- Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor (CVPR 2025 · similarity 0.88)
- Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.