Research map / Continual learning / Online and replay-based
Accurate Forgetting for Heterogeneous Federated Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AF-FCL, a generative-replay federated continual learning method that uses a conditional normalizing flow in feature space and density-based correlation weighting to selectively forget biased knowledge under a limitless task pool setting.
Problem
Federated continual learning where clients learn unrelated or even antagonistic tasks, so statistical heterogeneity and label noise cause biased or harmful features to be memorized in the replay memory and degrade performance.
Approach
Instead of indiscriminately replaying previous knowledge, a globally trained normalizing flow in the classifier's feature space generates replay features, which are re-weighted by their likelihood under a Gaussian fitted to the current client's local features ("accurate forgetting"), with feature-space distillation to limit drift.
Key results
- On EMNIST-LTP, AF-FCL reaches 47.5% average accuracy with 7.9% forgetting, versus 42.0% and 12.4% for the best baseline FedCIL.
- On MNIST-SVHN-F, AF-FCL reaches 68.1% accuracy versus 61.8% for GLFC.
- On CIFAR100 (N=8, T=6, C=10), AF-FCL reaches 23.8% accuracy versus 21.6% for the best baseline with forgetting 0.9%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add feature generative-replay (GR) (ablation) | AF-FCL · EMNIST-LTP | accuracy: 38.8$_{\pm 1.5}$ -> 47.5$_{\pm 0.3}$ | Helped |
| Add feature generative-replay (GR) (ablation) | AF-FCL · EMNIST-shuffle | accuracy: 70.8$_{\pm 0.7}$ -> 75.8$_{\pm 0.2}$ | Helped |
| Add knowledge distillation (KD) (ablation) | AF-FCL · EMNIST-LTP | accuracy: 44.3$_{\pm 0.6}$ -> 47.5$_{\pm 0.3}$ | Helped |
| Add knowledge distillation (KD) (ablation) | AF-FCL · EMNIST-shuffle | accuracy: 72.1$_{\pm 0.5}$ -> 75.8$_{\pm 0.2}$ | Helped |
| Add correlation estimation for accurate forgetting (AF) (ablation) | AF-FCL · EMNIST-LTP | accuracy: 41.8$_{\pm 0.3}$ -> 47.5$_{\pm 0.3}$ | Helped |
| Add correlation estimation for accurate forgetting (AF) (ablation) | AF-FCL · EMNIST-shuffle | accuracy: 71.0$_{\pm 0.9}$ -> 75.8$_{\pm 0.2}$ | Helped |
| Use AF-FCL instead of GLFC | federated continual learning · CIFAR100 | accuracy: 35.6$_{\pm 0.6}$ -> 36.3$_{\pm 0.4}$ | Helped |
| Use AF-FCL instead of GLFC | federated continual learning · MNIST-SVHN-F | accuracy: 61.8$_{\pm 0.8}$ -> 68.1$_{\pm 0.9}$ | Helped |
Surprising findings
- Several CL methods combined with FL (e.g., ACGAN-Replay+FedProx at 41.3%) outperform some dedicated FCL methods (FLwF2T at 40.1%, GLFC at 40.1%) on EMNIST-LTP.
- On the EMNIST-noisy setting, most CL and FCL baselines underperform plain FedAvg, suggesting memorizing noisy knowledge is harmful.
Limitations
- Correlation estimation relies on a single Gaussian with diagonal covariance per task, an approximation that may not capture complex feature distributions.
- Evaluation uses small-scale benchmarks (EMNIST, CIFAR100 subsets, MNIST-SVHN-F) with a three-layer CNN or ResNet-18 backbone; ImageNet results are only on a subset.
Experimental setup
Datasets: EMNIST CIFAR100 MNIST SVHN FashionMNIST ImageNet-Subset
Benchmarks: EMNIST-LTP EMNIST-shuffle CIFAR100 MNIST-SVHN-F EMNIST-noisy ImageNet-Subset
Metrics: average accuracy average forgetting
Baselines: FedAvg FedProx PODNet ACGAN-Replay FLwF2T FedCIL GLFC
Builds on: FedAvg FedProx ACGAN-Replay PODNet GLFC
Closest papers by meaning
- CAN: Leveraging Clients As Navigators for Generative Replay in Federated Continual Learning (ICML 2025 · similarity 0.92)
- Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyond (ICLR 2025 · similarity 0.92)
- Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor (CVPR 2025 · similarity 0.91)
- Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning (ICLR 2024 · similarity 0.91)
- Towards Efficient Replay in Federated Incremental Learning (CVPR 2024 · similarity 0.91)
- C$^2$Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learning (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.