Research map / Continual learning / Class-incremental
AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AnaCP, a replay-free analytic (closed-form, gradient-free) continual learner that adapts frozen pre-trained-model features via a contrastive projection layer with whitened, separation-enhanced class prototypes, then trains an ELM classifier on pseudo-replayed Gaussian features.
Problem
Class-incremental learning with pre-trained models: existing analytic methods use fixed PTM features and cannot adapt representations to CIL tasks without breaking closed-form updates, leaving them below joint training.
Approach
A closed-form contrastive projection layer whose target prototypes are the class means, whitened and SVD-perturbed to increase inter-class separation; the projection is updated incrementally via Gram and prototype statistics, so no gradient training and no forgetting in the adapted features.
Key results
- With DINO-v2, AnaCP reaches last accuracy 92.15 on CIFAR100 and 90.65 on Cars, exceeding joint fine-tuning on two datasets and within about 2% on the rest.
- AnaCP improves last accuracy over the best baseline by 1.03 to 3.17 points across datasets with DINO-v2 (relative error reduction 4.8% to 31.4%).
- With MoCo-v3, AnaCP still beats the best baseline but remains about 6% below joint fine-tuning; CIFAR100 training takes about 9 min 18 s with DINO-v2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add analytic contrastive projection to PTM features (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 88.13$_{\pm0.21}$ -> 91.86$_{\pm0.05}$ | Helped |
| add negative repulsion to class-mean targets (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 91.31$_{\pm0.03}$ -> 91.86$_{\pm0.05}$ | Helped |
| replace NCM with ELM classifier (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 91.86$_{\pm0.05}$ -> 92.15$_{\pm0.09}$ | Helped |
| increase generated pseudo-replay features per class from 20 to 100 (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 92.14$_{\pm0.06}$ -> 92.15$_{\pm0.09}$ | No clear effect |
| increase CP heads from 3 to 5 (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 92.15$_{\pm0.09}$ -> 92.20$_{\pm0.04}$ | No clear effect |
| reduce CP heads from 3 to 1 (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 92.15$_{\pm0.09}$ -> 91.99$_{\pm0.01}$ | Hurt |
| increase RP dimension from 5000 to 10000 (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 92.15$_{\pm0.09}$ -> 92.75$_{\pm0.10}$ | Helped |
| reduce RP dimension from 5000 to 1000 (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 92.15$_{\pm0.09}$ -> 90.64$_{\pm0.10}$ | Hurt |
| use class-specific covariance rather than shared covariance (ablation) | AnaCP with DINO-v2 · CIFAR100 | last accuracy: 92.15_{\pm0.09} -> 92.15_{\pm0.11} | No clear effect |
| use AnaCP instead of the strongest baseline | CIL with DINO-v2 · CIFAR100, ImageNet-R, CUB, TinyImageNet, Cars | last accuracy: 90.10 -> 92.15; 84.78 -> 86.60; 89.54 -> 90.57; 86.70 -> 87.35; 87.48 -> 90.65 | Helped |
| use AnaCP instead of the strongest baseline | CIL with MoCo-v3 · CIFAR100, ImageNet-R, CUB, TinyImageNet, Cars | last accuracy: 82.10 -> 83.70; 62.50 -> 67.30; 72.69 -> 78.61; 78.78 -> 79.32; 72.61 -> 78.70 | Helped |
Surprising findings
- With DINO-v2, AnaCP exceeds the joint fine-tuning upper bound on two of five datasets.
- Removing the contrastive projection drops accuracy by roughly 3 to 20 points, larger than the classifier change.
Limitations
- With a weaker PTM (MoCo-v3) AnaCP does not match joint fine-tuning, about 6% gap.
- Relies on a strong pre-trained model; first-session adaptation of the PTM is still needed.
- Evaluated only on CIL, with TIL and DIL left as future extensions.
Experimental setup
Datasets: CIFAR100 ImageNet-R CUB TinyImageNet Cars
Benchmarks: CIFAR100 ImageNet-R CUB TinyImageNet Cars
Metrics: last accuracy average incremental accuracy running time memory
Baselines: L2P DualPrompt CODA-Prompt SLCA SimpleCIL SLDA KLDA GACL APER FeCAM RanPAC
Builds on: extreme learning machine RanPAC SimpleCIL SLDA KLDA GACL
Closest papers by meaning
- Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning (AAAI 2025 · similarity 0.89)
- GACL: Exemplar-Free Generalized Analytic Continual Learning (NeurIPS 2024 · similarity 0.88)
- Continual Learning Using a Kernel-Based Method Over Foundation Models (AAAI 2025 · similarity 0.88)
- Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning (NeurIPS 2024 · similarity 0.87)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.87)
- Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.