Research map / Federated learning and privacy / Personalized and heterogeneous FL
Traceable Federated Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Traceable Federated Continual Learning (TFCL) and TagFed, which prunes sub-models per task to trace and retrain repeated tasks and aggregates knowledge group-wise via server-side distillation, evaluated on a self-constructed CIFAR-100/ImageNetSubset/TinyImageNet benchmark.
Problem
Federated continual learning assumes each incoming task is new, but in practice tasks repeat across time and clients, and existing FCL methods perform poorly when tasks recur.
Approach
Decompose the model into marked pruning-based sub-models per task so repeated tasks can be located and retrained, and aggregate knowledge through several server-side model groups using logit/feature-based distillation instead of weight averaging.
Key results
- TagFed reaches 78.35% accuracy at 10 tasks on CIFAR-100, versus 56.05% without TTL, and outperforms PackNet+FedAvg, GLFC, FedWeIT and FedKNOW across datasets.
- TagFed uses roughly 50% less communication cost than compared methods, with offloading cost of 0.05MB versus 11.7MB for PackNet+FedAvg.
- Under dFIL-based noise, TagFed reaches 70.3%-74.1% on CIFAR-100 across 2-5 tasks, above all baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TagFed instead of the compared federated continual learning baselines | CIFAR-100, ImageNetSubset, and TinyImageNet benchmark · CIFAR-100, ImageNetSubset, and TinyImageNet | final accuracy | Helped |
| Reduce offloading by sending prediction vectors rather than full model information | TagFed · CIFAR-100 | offloading cost: 11.70MB -> 0.05MB; 11.70MB -> 0.05MB; 26.20MB -> 0.05MB; 11.70MB -> 0.05MB | Helped |
| Add traceable task learning (TTL) (ablation) | TagFed · CIFAR-100, task scale 2 | accuracy: 69.20% -> 71.40% | Helped |
| Add traceable task learning (TTL) (ablation) | TagFed · CIFAR-100, task scale 4 | accuracy: 59.05% -> 75.39% | Helped |
| Add traceable task learning (TTL) (ablation) | TagFed · CIFAR-100, task scale 6 | accuracy: 55.62% -> 76.36% | Helped |
| Add traceable task learning (TTL) (ablation) | TagFed · CIFAR-100, task scale 8 | accuracy: 55.65% -> 77.18% | Helped |
| Add traceable task learning (TTL) (ablation) | TagFed · CIFAR-100, task scale 10 | accuracy: 56.05% -> 78.35% | Helped |
| Add group-wise knowledge aggregation (GKA) (ablation) | TagFed · CIFAR-100, task scale 2 | accuracy: 71.00% -> 71.40% | Helped |
| Add group-wise knowledge aggregation (GKA) (ablation) | TagFed · CIFAR-100, task scale 4 | accuracy: 73.43% -> 75.39% | Helped |
| Add group-wise knowledge aggregation (GKA) (ablation) | TagFed · CIFAR-100, task scale 6 | accuracy: 75.25% -> 76.36% | Helped |
| Add group-wise knowledge aggregation (GKA) (ablation) | TagFed · CIFAR-100, task scale 8 | accuracy: 76.21% -> 77.18% | Helped |
| Add group-wise knowledge aggregation (GKA) (ablation) | TagFed · CIFAR-100, task scale 10 | accuracy: 75.76% -> 78.35% | Helped |
Surprising findings
- Traditional FCL baselines such as GLFC degrade noticeably when repetitive tasks are introduced, with accuracy dropping over rounds (Figure 2).
Limitations
- Benchmark is manually constructed by the authors because no existing TFCL benchmark exists, limiting comparability.
- Parameter storage for sub-models is roughly 10% of the model per task at clients, with sparsity set to 0.1 to fit all tasks in one model.
- Privacy protection relies on added noise that slightly reduces accuracy.
Experimental setup
Datasets: CIFAR-100 ImageNetSubset TinyImageNet
Benchmarks: CIFAR-100 ImageNetSubset TinyImageNet
Metrics: accuracy communication cost computational cost
Baselines: PackNet+FedAvg GLFC FedWeIT FedKNOW
Builds on: PackNet FedAvg GLFC FedWeIT FedKNOW
Closest papers by meaning
- Federated Continual Learning via Prompt-based Dual Knowledge Transfer (ICML 2024 · similarity 0.89)
- Accurate Forgetting for Heterogeneous Federated Continual Learning (ICLR 2024 · similarity 0.88)
- Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learning (NeurIPS 2025 · similarity 0.88)
- Resource-Constrained Federated Continual Learning: What Does Matter? (NeurIPS 2025 · similarity 0.88)
- HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning (NeurIPS 2026 · similarity 0.87)
- CAN: Leveraging Clients As Navigators for Generative Replay in Federated Continual Learning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.