Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Continual Learning via Prompt-based Dual Knowledge Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Powder, a prompt-based federated continual learning method that exploits dual (spatial and temporal) knowledge transfer via task-correlation-based two-step prompt aggregation and a dual distillation loss.
Problem
Federated continual learning methods mainly avoid inter-task interference and do not exploit positive knowledge transfer across tasks learned by different clients at different times, while also facing communication and privacy costs from transmitting large prompt pools.
Approach
Estimate task and class correlations from prompt selection weights, aggregate prompts on the server by correlation (first step) and send only top-k related prompts to clients (second step), and retain transferred knowledge with a correlation-weighted dual distillation loss.
Key results
- On ImageNet-R (C=5, r=15), Powder reaches 84.08% AIA versus 75.14% for Fed-CODAP and is the only method with positive Combined Transfer (6.04%).
- Powder cuts communication overhead to 493.08MB from 815.63MB for Fed-CODAP/FedCPrompt (roughly 20-40% reduction) while improving accuracy by 9-12%.
- On DomainNet, Powder achieves 77.28% AIA and positive CT of 1.40%, while all baselines have negative CT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Powder instead of GLFC-ViT | ViT-based FCL · ImageNet-R | AIA (%): 75.21(1.05) -> 84.08(0.56) | Helped |
| use Powder instead of Fed-CODAP | ViT-based FCL · ImageNet-R | CT (%): −1.18(1.62) -> 6.04(0.50) | Helped |
| use Powder instead of Fed-CODAP | ViT-based FCL · DomainNet | AIA (%): 72.84(0.40) -> 77.28(0.18) | Helped |
| use Powder instead of Fed-CODAP | ViT-based FCL · DomainNet | CT (%): −0.15(0.29) -> 1.40(0.19) | Helped |
| add first-step aggregation (FUS) (ablation) | Powder · ImageNet-R | AIA (%): 79.88(0.83) -> 84.08(0.56) | Helped |
| add first-step aggregation (FUS) (ablation) | Powder · ImageNet-R | CT (%): −0.12(0.90) -> 6.04(0.50) | Helped |
| select top-k related prompts instead of randomly selected prompts (ablation) | Powder · ImageNet-R | AIA (%): 79.23(0.87) -> 84.08(0.56) | Helped |
| select top-k related prompts instead of randomly selected prompts (ablation) | Powder · ImageNet-R | CT (%): −0.66(0.78) -> 6.04(0.50) | Helped |
| add dual distillation loss (ablation) | Powder · ImageNet-R | AIA (%): 76.38(0.89) -> 84.08(0.56) | Helped |
| add dual distillation loss (ablation) | Powder · ImageNet-R | CT (%): −4.37(0.92) -> 6.04(0.50) | Helped |
| use dual rather than classic knowledge distillation (ablation) | Powder · ImageNet-R | AIA (%): 81.35(0.94) -> 84.08(0.56) | Helped |
| use dual rather than classic knowledge distillation (ablation) | Powder · ImageNet-R | CT (%): 1.60(0.48) -> 6.04(0.50) | Helped |
Surprising findings
- Existing rehearsal-free and rehearsal-based FCL baselines show negative forward and combined transfer, indicating negative inter-task interference outweighs positive transfer.
Limitations
- Baselines non-ViT methods had to be adapted to ViT and combined with prompt-tuning, making comparisons dependent on re-implementation choices.
- Benchmark uses only two image datasets (ImageNet-R and DomainNet) with simulated asynchronous task assignment.
- Method relies on a sensitivity hyperparameter p and k that require validation-set tuning.
Experimental setup
Datasets: ImageNet-R DomainNet
Benchmarks: ImageNet-R DomainNet
Metrics: Average Incremental Accuracy Forgetting Measure Forward Transfer Backward Transfer Combined Transfer communication overhead (MB)
Baselines: FedWEIT CFeD GLFC FedSpace Fed-L2P Fed-Dual Fed-CODAP FedCPrompt
Builds on: CODAPrompt L2P DualPrompt FedAvg FedNTD
Closest papers by meaning
- C$^2$Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learning (NeurIPS 2025 · similarity 0.90)
- Traceable Federated Continual Learning (CVPR 2024 · similarity 0.89)
- Accurate Forgetting for Heterogeneous Federated Continual Learning (ICLR 2024 · similarity 0.89)
- Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learning (NeurIPS 2025 · similarity 0.88)
- CAN: Leveraging Clients As Navigators for Generative Replay in Federated Continual Learning (ICML 2025 · similarity 0.87)
- Resource-Constrained Federated Continual Learning: What Does Matter? (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.