Research map / Continual learning / Class-incremental
Text-Enhanced Data-free Approach for Federated Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LANDER, which uses pretrained label-text embeddings as anchors both to organize client feature embeddings (via a Bounding Loss) and to guide data-free synthesis of old-task samples for federated class-incremental learning.
Problem
Federated class-incremental learning suffers from catastrophic forgetting, and data-free knowledge transfer (DFKT) generates low-quality synthetic samples because the teacher's high-confidence regions are disorganized and lack shared anchors between clients and server.
Approach
Use CLIP label-text embeddings as shared anchors: client features are pulled into a fixed radius around them via a Bounding Loss (instead of MSE collapse), and the data generator takes these embeddings as input with a noisy layer, plus learnable data statistics for normalization.
Key results
- On CIFAR-100 with 5 tasks IID, LANDER reaches 52.60% average accuracy versus 36.31% for TARGET.
- On Tiny-ImageNet 5 tasks IID, LANDER achieves 30.29% accuracy versus 26.25% for TARGET.
- Ablation: the Bounding Loss radius r=0.015 gives the best accuracy; the paper reports that removing it (r=0) lowers accuracy in high Dirichlet settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LANDER instead of TARGET | ResNet18 FCIL · CIFAR-100, IID, T=5 | average accuracy (%): 36.31 -> 52.60 | Helped |
| use LANDER instead of TARGET | ResNet18 FCIL · Tiny-ImageNet, IID, T=5 | average accuracy (%): 26.25 -> 30.29 | Helped |
| use LANDER instead of TARGET | ResNet18 FCIL · ImageNet, IID, Task 5 | average accuracy (%): 31.83 -> 43.24 | Helped |
| use LANDER instead of TARGET | ResNet18 FCIL · CIFAR-100, IID, T=5 | forgetting: 32.23 -> 18.03 | Helped |
| remove LTE constraints from data-free generation (ablation) | LANDER · CIFAR-100, T=5 and T=10 | accuracy | Hurt |
| remove the noisy layer and LTE input in data-free generation (ablation) | LANDER · CIFAR-100, T=5 and T=10 | accuracy | Hurt |
| use Bounding Loss with radius r=0.015 rather than r=0 (ablation) | LANDER · CIFAR-100, heterogeneous federated setting | incremental accuracy | Helped |
| use learnable data stats rather than random stats or no normalization (ablation) | LANDER data-free generation · CIFAR-100, T=5 and T=10 | accuracy | Helped |
| replace the class-name prompt P2 with P1 (ablation) | LANDER · CIFAR-100, IID, T=5 | accuracy: 52.60 -> 52.45 | Hurt |
| replace the class-name prompt P2 with label-index prompt P3 (ablation) | LANDER · CIFAR-100, IID, T=5 | accuracy: 52.60 -> 50.12 | Hurt |
| replace CLIP with Doc2Vec as the text encoder (ablation) | LANDER · CIFAR-100, NIID(0.5), T=5 | accuracy: 48.23 -> 48.21 | Hurt |
| replace CLIP with SBERT as the text encoder (ablation) | LANDER · CIFAR-100, NIID(0.5), T=5 | accuracy: 48.23 -> 48.18 | Hurt |
Surprising findings
- Pulling features exactly onto the label-text anchor (MSE) worsens data imbalance and embedding overlap in federated settings, motivating the bounding radius instead.
Limitations
- Gains on CIFAR-100 are reported only against a small set of baselines; several baselines' results are copied from TARGET's paper.
- Requires a pretrained text encoder to produce label embeddings, and performance depends on the prompt template.
Experimental setup
Datasets: CIFAR-100 Tiny-ImageNet ImageNet
Benchmarks: CIFAR-100 Tiny-ImageNet ImageNet
Metrics: average accuracy forgetting measure
Baselines: Finetune FedEWC FedWeIT FedLwF TARGET
Builds on: TARGET DFKT (data-free knowledge transfer) CLIP-based label text prompting (Inspired by [53]) FedLwF Dynamic-ratio adaptive loss weighting from [19]
Closest papers by meaning
- Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting (ICLR 2025 · similarity 0.85)
- FedWon: Triumphing Multi-domain Federated Learning Without Normalization (ICLR 2024 · similarity 0.85)
- Splitting with Importance-aware Updating for Heterogeneous Federated Learning with Large Language Models (ICML 2025 · similarity 0.85)
- FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · similarity 0.85)
- Exploring Vacant Classes in Label-Skewed Federated Learning (AAAI 2025 · similarity 0.85)
- FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.