Research map / Continual learning / Class-incremental
Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EASE, an exemplar-free pre-trained-model class-incremental learning method that adds a lightweight adapter per task to form expandable subspaces, and synthesizes old-class prototypes in new subspaces via semantic similarity.
Problem
Class-incremental learning with pre-trained models suffers from forgetting when updating, and expandable-backbone methods are too costly and need exemplars; exemplar-free PTM-based CIL needs a way to add task-specific capacity without conflict and to map expanding features to old classes.
Approach
Freeze the pre-trained ViT and train one bottleneck adapter per incremental task to create task-specific subspaces; concatenate features from all adapters, synthesize missing old-class prototypes in the newest subspace via class-similarity recombination from the co-occurrence subspace, and reweight the matching subspace's logits at inference.
Key results
- On ImageNet-R B0 Inc5 (ViT-B/16-IN21K) EASE attains the best results among seven benchmarks in Table 1, e.g., 78.31 average and 70.58 last on ImageNet-R... (reported as IN-R column)
- On VTAB B0 Inc10 EASE reaches 93.61 average and 93.55 last accuracy, versus 87.47 for the runner-up ADAM+Finetune-type baselines.
- EASE outperforms exemplar-based methods such as FOSTER and DER on CIFAR and ImageNet-R B0 settings while using zero exemplars (e.g., 92.35 vs 89.87 average on CIFAR B0 Inc10 for FOSTER).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EASE instead of ADAM + Adapter | PTM-based class-incremental learning · ImageNet-R B0 Inc5 | average accuracy: 72.35 -> 78.31 | Helped |
| Use EASE instead of ADAM + VPT-S | PTM-based class-incremental learning · VTAB B0 Inc10 | last accuracy: 85.36 -> 93.55 | Helped |
| Use EASE instead of FOSTER with 20 exemplars per class | Pre-trained-model CIL · CIFAR B0 Inc10 | last accuracy: 84.91 -> 87.76 | Helped |
| Add task-specific adapters to the pre-trained model (ablation) | EASE · ImageNet-R B0 Inc20 | accuracy | Helped |
| Add semantic-guided prototype complement rather than using only diagonal classifier components (ablation) | EASE with task-specific adapters · ImageNet-R B0 Inc20 | accuracy | Helped |
| Add subspace reweighting to the prototype-complement model (ablation) | EASE with prototype complement · ImageNet-R B0 Inc20 | accuracy | Helped |
| Use similarity-based prototype complement instead of linear regression or optimal transport (ablation) | EASE · ImageNet-R, ImageNet-A, and ObjectNet | last accuracy | Helped |
Surprising findings
- EASE without any exemplars is competitive with or better than exemplar-based CIL methods such as iCaRL, DER, FOSTER and MEMO when all use the same PTM.
Limitations
- Adapters still add model storage that grows linearly with the number of tasks.
- Compression of adapters is left as future work.
Experimental setup
Datasets: CIFAR100 CUB200 ImageNet-R ImageNet-A ObjectNet OmniBenchmark VTAB
Benchmarks: CIFAR100 CUB200 ImageNet-R ImageNet-A ObjectNet OmniBenchmark VTAB
Metrics: average incremental accuracy last-stage accuracy
Baselines: L2P DualPrompt CODA-Prompt SimpleCIL ADAM LwF SDC iCaRL DER FOSTER MEMO
Builds on: Adapter tuning ADAM expandable networks (DER/FOSTER) prototype classifier
Closest papers by meaning
- Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning (CVPR 2025 · similarity 0.87)
- Harnessing Neural Unit Dynamics for Effective and Scalable Class-Incremental Learning (ICML 2024 · similarity 0.87)
- Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.87)
- Knowledge Memorization and Rumination for Pre-trained Model-based Class-Incremental Learning (CVPR 2025 · similarity 0.86)
- Reducing Class-wise Confusion for Incremental Learning with Disentangled Manifolds (CVPR 2025 · similarity 0.86)
- Compress to One Point: Neural Collapse for Pre-Trained Model-Based Class-Incremental Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.