Research map / Continual learning / Online and replay-based
Resource-Constrained Federated Continual Learning: What Does Matter?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks federated continual learning (FCL) methods under constrained memory buffer, computational budget, and label rate across six datasets in Class-IL and Domain-IL scenarios, finding existing FCL methods largely fail under limited resources.
Problem
Existing federated continual learning methods assume unrestricted training resources, but edge devices are constrained by storage (memory buffer), computation budget, and label rate; it is unclear how FCL methods perform under such constraints.
Approach
Revisits FCL under explicit resource constraints (memory buffer size M, gradient-step budget B, label rate R) via a large-scale benchmark, dissecting FCL methods into four techniques (sample caching, data synthesis, knowledge distillation, network extension) and analyzing each under constraints.
Key results
- Under extremely limited resources, accuracy drops sharply for all methods; e.g., FedAvg final accuracy on CIFAR-10 falls from 40.94% (sufficient) to 10.11%.
- Under sufficient resources, FedCIL and AF-FCL perform well on CIFAR-10 and Digit-10 (e.g., AF-FCL 44.95% final on CIFAR-10) but degrade on CIFAR-100 and Tiny-ImageNet.
- Across experiments totaling over 1,000 GPU hours, the paper finds existing FCL approaches fail to achieve expected performance under constrained resources.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Simultaneously restrict memory buffer, computational budget, and label rate | Existing FCL methods · Six datasets across Class-IL and Domain-IL | accuracy | Hurt |
| Increase memory buffer from a limited amount to a modest amount (ablation) | FCL methods using cached, synthetic, or distillation samples · Six datasets | accuracy | Helped |
| Increase memory buffer beyond the performance plateau (ablation) | FCL methods using memory buffers · Six datasets | accuracy | No clear effect |
| Restrict computational budget (ablation) | FCL methods requiring complex computation · Six datasets | accuracy | Hurt |
| Increase label rate when it is below 70% (ablation) | Most FCL methods · Six datasets | accuracy | Helped |
| Increase label rate (ablation) | FCL methods on Digit-10 · Digit-10 | accuracy | No clear effect |
| Increase the number of Class-IL tasks (ablation) | Evaluated FCL methods · Class-IL datasets | accuracy | Hurt |
| Increase memory buffer for sample caching (ablation) | Sample-caching FCL methods · Class-IL and Domain-IL datasets | accuracy | Mixed |
| Increase memory buffer capacity for data synthesis (ablation) | Generative-replay FCL methods · Class-IL and Domain-IL datasets | accuracy | Helped |
| Increase memory buffer beyond the sufficient capacity for data synthesis (ablation) | Generative-replay FCL methods · Class-IL and Domain-IL datasets | accuracy | No clear effect |
| Increase computational budget for knowledge distillation (ablation) | Knowledge-distillation FCL methods · Class-IL and Domain-IL datasets | accuracy | Helped |
| Increase memory buffer for knowledge distillation (ablation) | Knowledge-distillation FCL methods · Class-IL and Domain-IL datasets | accuracy | No clear effect |
Surprising findings
- Simple FedAvg shows remarkable resilience to computational budget restrictions, comparable to FOT.
- Network extension methods relying on task-ID knowledge fail when task boundaries are unknown at inference.
Limitations
- FOT could not be run stably with ResNet18, so AlexNet was used, confounding backbone comparisons.
- Network extension methods were not experimentally evaluated due to reliance on task-ID assumptions.
- Each setting run only twice, with limited statistical reliability.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet Digit-10 Office-31 Office-Caltech-10
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet Digit-10 Office-31 Office-Caltech-10
Metrics: final accuracy A(f) average accuracy
Baselines: FedAvg FedProx FedAvg+ER FedProx+ER FedCIL TARGET AF-FCL Re-Fed GLFC FOT CFeD FedWeIT
Builds on: FedAvg FedProx FedCIL AF-FCL Re-Fed FOT FedWeIT TARGET
Closest papers by meaning
- HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning (NeurIPS 2026 · similarity 0.89)
- Traceable Federated Continual Learning (CVPR 2024 · similarity 0.88)
- CAN: Leveraging Clients As Navigators for Generative Replay in Federated Continual Learning (ICML 2025 · similarity 0.87)
- Accurate Forgetting for Heterogeneous Federated Continual Learning (ICLR 2024 · similarity 0.87)
- Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyond (ICLR 2025 · similarity 0.87)
- Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.