Research map / Continual learning / Online and replay-based
FedSSI: Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedSSI, a rehearsal-free regularization method for continual federated learning that adapts synaptic intelligence using a personalized surrogate model that blends local and global information to handle data heterogeneity.
Problem
Continual federated learning suffers catastrophic forgetting under data heterogeneity; existing CFL methods rely on rehearsal (memory buffers or generated samples) which costs memory and risks privacy, and regularization methods like synaptic intelligence degrade on non-IID client data.
Approach
Computes synaptic intelligence importance scores using a personalized surrogate model trained locally from the aggregated global model with a proximal-style term controlled by lambda, so importance reflects both local and global data distributions; no rehearsal or extra communication is needed.
Key results
- FedSSI reaches 42.58% final accuracy on CIFAR10 (alpha=0.1) versus 40.18% for the best baseline FOT, per the main table.
- FedSSI reaches 56.80% final accuracy on Tiny-ImageNet versus 54.87% for FOT (the best baseline in the table).
- Plain FL+SI is the strongest regularization baseline under IID-like settings but drops sharply under non-IID data according to the observation section.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedSSI rather than the strongest listed baseline | CFL with ResNet18 · CIFAR10 | final accuracy: 40.18 -> 42.58 | Helped |
| Use FedSSI rather than the strongest listed baseline | CFL with ResNet18 · CIFAR100 | final accuracy: 36.15 -> 37.96 | Helped |
| Use FedSSI rather than the strongest listed baseline | CFL with ResNet18 · Tiny-ImageNet | final accuracy: 37.23 -> 40.56 | Helped |
| Use FedSSI rather than the strongest listed baseline | CFL with ResNet18 · Digit10 | final accuracy: 69.79 -> 72.09 | Helped |
| Use FedSSI rather than the strongest listed baseline | CFL with ResNet18 · Office31 | final accuracy: 53.10 -> 55.28 | Helped |
| Use FedSSI rather than the strongest listed baseline | CFL with ResNet18 · Office-Caltech-10 | final accuracy: 60.30 -> 62.57 | Helped |
| Set λ=0.2 instead of λ=0.8 (ablation) | FedSSI · CIFAR10, α=0.1 | test accuracy: 39.93 -> 42.58 | Helped |
| Set λ=0.8 instead of λ=0.2 (ablation) | FedSSI · CIFAR10, α=100 | test accuracy: 61.28 -> 63.92 | Helped |
| Set λ=0.2 instead of λ=0.8 (ablation) | FedSSI · CIFAR100, α=0.1 | test accuracy: 26.41 -> 28.43 | Helped |
| Set λ=0.8 instead of λ=0.2 (ablation) | FedSSI · CIFAR100, α=100 | test accuracy: 44.79 -> 45.67 | Helped |
| Set λ=0.2 instead of λ=0.8 (ablation) | FedSSI · Office31, α=0.1 | test accuracy: 46.96 -> 49.64 | Helped |
| Set λ=0.8 instead of λ=0.2 (ablation) | FedSSI · Office31, α=100 | test accuracy: 67.03 -> 68.45 | Helped |
Surprising findings
- Plain FL+SI, which performs best among regularization methods with IID data, declines the most under heterogeneous client data.
- Regularization-based methods and rehearsal-based CFL methods with a 300-sample buffer perform close to the FedAvg baseline when heterogeneity is high (alpha=0.1).
Limitations
- FedSSI uses more communication rounds than some baselines in the reported round-count analysis (e.g. 225 rounds on Digit10 vs 208 for FedAvg).
- Gains on Digit10 are small, which the authors attribute to simple one-channel data.
- The relation between heterogeneity alpha and the hyper-parameter lambda is only supported empirically, not by a closed-form rule.
Experimental setup
Datasets: CIFAR10 CIFAR100 Tiny-ImageNet Digit10 Office31 Office-Caltech-10
Benchmarks: CIFAR10 CIFAR100 Tiny-ImageNet Digit10 Office31 Office-Caltech-10
Metrics: final accuracy A(f) average accuracy communication rounds
Baselines: FedAvg FedProx FL+LwF FL+EWC FL+OGD FL+SI Re-Fed FedCIL GLFC FOT FedWeIT
Builds on: Synaptic Intelligence FedAvg FedProx Elastic Weight Consolidation Learning without Forgetting
Closest papers by meaning
- Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning (ICLR 2024 · similarity 0.88)
- Accurate Forgetting for Heterogeneous Federated Continual Learning (ICLR 2024 · similarity 0.88)
- Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor (CVPR 2025 · similarity 0.86)
- CAN: Leveraging Clients As Navigators for Generative Replay in Federated Continual Learning (ICML 2025 · similarity 0.86)
- HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning (NeurIPS 2026 · similarity 0.86)
- Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.