Research map / Continual learning / Online and replay-based
Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedTA, a federated continual learning method that freezes a pre-trained ViT and mixes trainable tail anchors with output features, plus input enhancement, selective knowledge fusion, and best global prototype selection, to counter spatial-temporal catastrophic forgetting.
Problem
Federated continual learning suffers from spatial (non-IID across clients) and temporal (across tasks) data heterogeneity, which causes parameter-level and output-level catastrophic forgetting.
Approach
Freeze a pre-trained ViT to prevent parameter forgetting and mix learnable tail anchors (selected by key similarity) with frozen output features, pulling features toward fixed server-selected global class prototypes to prevent output forgetting.
Key results
- On CIFAR-100, FedTA reaches 96.1% on task 1 and about 94% on tasks 2-4 of the aggregated global model's local accuracy, outperforming FedMGP (90.2% on task 1).
- On ImageNet-R, FedTA achieves 81.5, 78.8, 79.2, 80.6, 85.0 across the five tasks, above FedMGP (77.3, 76.8, 78.0, 75.6, 75.4).
- FedTA keeps both temporal and spatial knowledge retention at around 98%, while baselines such as TARGET and MFCL perform poorly on temporal retention.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FedTA instead of FedMGP | Federated continual learning · CIFAR-100, Task 5 | accuracy (%): 82.2 -> 89.4 | Helped |
| use FedTA instead of FedMGP | Federated continual learning · ImageNet-R, Task 5 | accuracy (%): 75.4 -> 85.0 | Helped |
| add Tail Anchor (ablation) | FedTA · CIFAR-100, Task 5 | accuracy (%): 72.3 -> 89.4 | Helped |
| add Tail Anchor (ablation) | FedTA · ImageNet-R, Task 5 | accuracy (%): 63.2 -> 85.0 | Helped |
| add Selective Input Knowledge Fusion (ablation) | FedTA · CIFAR-100, Task 5 | accuracy (%): 89.1 -> 89.4 | Helped |
| add Selective Input Knowledge Fusion (ablation) | FedTA · ImageNet-R, Task 5 | accuracy (%): 81.7 -> 85.0 | Helped |
| add Best Global Prototype Selection (ablation) | FedTA · CIFAR-100, Task 5 | accuracy (%): 88.6 -> 89.4 | Helped |
| add Best Global Prototype Selection (ablation) | FedTA · ImageNet-R, Task 5 | accuracy (%): 79.1 -> 85.0 | Helped |
Surprising findings
- Replay-based baselines (TARGET, FedLwF, MFCL) perform extremely poorly, which the authors attribute to low-quality pseudo-samples from large 224x224 images.
- Selective input knowledge fusion sometimes falls below plain weighted averaging on ImageNet-R, attributed to insufficient surrogate data.
Limitations
- Server needs a small surrogate dataset, and selective fusion underperforms weighted averaging when this data is small (ImageNet-R).
- Uploaded class prototypes may leak original features; the authors suggest random mixing as a fix but do not evaluate it.
- Relies on a frozen pre-trained ViT, so performance depends on the pre-trained backbone.
Experimental setup
Datasets: CIFAR-100 ImageNet-R
Benchmarks: CIFAR-100 ImageNet-R
Metrics: average accuracy on local test sets temporal knowledge retention spatial knowledge retention
Baselines: FedAvg FedProx FedNova FedLwF FedViT FedL2P FedDualP GLFC TARGET MFCL FedMGP
Builds on: FedViT GLFC FedMGP L2P DualPrompt
Closest papers by meaning
- Accurate Forgetting for Heterogeneous Federated Continual Learning (ICLR 2024 · similarity 0.91)
- HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning (NeurIPS 2026 · similarity 0.88)
- Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning (ICLR 2024 · similarity 0.88)
- Towards Efficient Replay in Federated Incremental Learning (CVPR 2024 · similarity 0.88)
- Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyond (ICLR 2025 · similarity 0.87)
- FedSSI: Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligence (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.