Research map / Continual learning / Class-incremental
Compress to One Point: Neural Collapse for Pre-Trained Model-Based Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PTM-CIL method that freezes a Simplex ETF classifier (from Neural Collapse theory) and trains a task-related adapter plus feature compression module to pull features onto the fixed class prototypes across incremental steps.
Problem
Class-incremental learning with pre-trained models: generalized PTM features are not discriminative for out-of-distribution incremental tasks, and class relationships cannot be optimally constructed when classes arrive in separate steps.
Approach
Pre-assign a frozen Simplex Equiangular Tight Frame classifier as the optimal class structure for all seen classes, then use a task-related adapter and a feature compression module (with feature replay and an optimal structural alignment loss) to compress features onto the assigned ETF prototypes, guided by Neural Collapse.
Key results
- With ViT-B/16-IN21K, the FM variant reaches average incremental accuracy 94.41 on CIFAR100 B0 Inc5 versus 91.51 for EASE, the prior best.
- On ImageNet-R B0 Inc5 the FM variant reports 81.64 average accuracy versus 78.31 for EASE, with gains of 3.33 points reported in the text.
- The CM variant (no stored features beyond class statistics) reaches second-best performance on most datasets, e.g. 94.01 on CIFAR100 average accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Ours/with FM instead of the strongest listed baseline, RanPAC | PTM-CIL with ViT-B/16-IN21K · CIFAR100 B0 Inc5 | average incremental accuracy: 93.51 -> 94.41 | Helped |
| use Ours/with FM instead of the strongest listed baseline, RanPAC | PTM-CIL with ViT-B/16-IN21K · CUB B0 Inc10 | last accuracy: 89.40 -> 89.14 | Hurt |
| use Ours/with FM instead of the strongest listed baseline, EASE | PTM-CIL with ViT-B/16-IN21K · ImageNet-R B0 Inc5 | average incremental accuracy: 78.31 -> 81.64 | Helped |
| use Ours/with FM instead of the strongest listed baseline, EASE | PTM-CIL with ViT-B/16-IN21K · ImageNet-A B0 Inc20 | average incremental accuracy: 65.34 -> 67.43 | Helped |
| use Ours/with FM instead of the strongest listed baseline, RanPAC | PTM-CIL with ViT-B/16-IN21K · ObjectNet B0 Inc10 | average incremental accuracy: 71.67 -> 75.23 | Helped |
| use Ours/with FM instead of the strongest listed baseline, RanPAC | PTM-CIL with ViT-B/16-IN21K · OmniBenchmark B0 Inc30 | average incremental accuracy: 85.95 -> 87.23 | Helped |
| use Ours/with FM instead of the strongest listed baseline, EASE | PTM-CIL with ViT-B/16-IN21K · VTAB B0 Inc10 | last accuracy: 93.55 -> 90.60 | Hurt |
| add Task-Related Adaptation (ablation) | PTM-CIL on CIFAR100 B0 Inc5 · CIFAR100 B0 Inc5 | accuracy: 90.22 -> 90.62 | Helped |
| add the ETF classifier (ablation) | PTM-CIL on CIFAR100 B0 Inc5 · CIFAR100 B0 Inc5 | accuracy: 90.62 -> 91.32 | Helped |
| add the Feature Compression Module (ablation) | PTM-CIL on CIFAR100 B0 Inc5 · CIFAR100 B0 Inc5 | accuracy: 91.32 -> 93.43 | Helped |
| add Optimal Structural Alignment (ablation) | PTM-CIL on CIFAR100 B0 Inc5 · CIFAR100 B0 Inc5 | accuracy: 93.43 -> 94.41 | Helped |
| increase replayed feature count beyond 256 (ablation) | Ours-CM on CIFAR100 B0 Inc5 · CIFAR100 B0 Inc5 | performance | No clear effect |
Surprising findings
- Removing the fixed ETF classifier (using random fixed classifier weights) noticeably degrades performance, indicating non-optimal structure hurts feature adaptation.
Limitations
- Feature Memory variant requires storing features of all samples, increasing memory cost
- Performance on VTAB (small dataset with few classes) is not best at last-step accuracy
- Frozen backbone and adapter after base step rely on the base-step adaptation quality
Experimental setup
Datasets: CIFAR100 CUB ImageNet-R ImageNet-A ObjectNet OmniBenchmark VTAB
Benchmarks: CIFAR100 CUB ImageNet-R ImageNet-A ObjectNet OmniBenchmark VTAB
Metrics: average incremental accuracy last-step accuracy
Baselines: Finetune Finetune Adapter LwF SDC L2P DualPrompt CODA-Prompt SimpleCIL ADAM EASE RanPAC
Builds on: EASE ADAM SimpleCIL Neural Collapse
Closest papers by meaning
- eTag: Class-Incremental Learning via Embedding Distillation and Task-Oriented Generation (AAAI 2024 · similarity 0.88)
- Harnessing Neural Unit Dynamics for Effective and Scalable Class-Incremental Learning (ICML 2024 · similarity 0.87)
- Knowledge Memorization and Rumination for Pre-trained Model-based Class-Incremental Learning (CVPR 2025 · similarity 0.87)
- Mixture of Noise for Pre-Trained Model-Based Class-Incremental Learning (NeurIPS 2025 · similarity 0.87)
- AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projection (NeurIPS 2025 · similarity 0.87)
- PEARL: Input-Agnostic Prompt Enhancement with Negative Feedback Regulation for Class-Incremental Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.