Research map / Continual learning / Class-incremental
GACL: Exemplar-Free Generalized Analytic Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GACL, an exemplar-free generalized class-incremental learner using closed-form analytic least-squares with a frozen DeiT-S backbone, achieving a weight-invariant (joint-training-equivalent) solution under Si-Blurry settings.
Problem
Class-incremental learning under generalized settings (GCIL) where tasks have mixed/reappearing classes and uneven sample sizes, without storing exemplars, suffers from catastrophic forgetting and task-recency bias.
Approach
Extends analytic continual learning (ACIL) to GCIL by splitting incoming labels into exposed (previously seen) and unexposed classes and adding an exposed-class label gain (ECLG) term, so recursive weight updates exactly match joint training regardless of how data is split across tasks.
Key results
- On Tiny-ImageNet, GACL reaches 63.14% AUC, 69.32% Avg and 62.68% Last accuracy, beating the best EFCIL baseline by 13.97, 21.39 and 9.55 points.
- On CIFAR-100 (r_B=10%), GACL achieves 57.99% AUC / 56.24% Avg / 70.31% Last, exceeding SLDA by 4.99, 6.15 and 8.52 points.
- Removing the ECLG module drops CIFAR-100 Last accuracy from 70.31% to 47.30% at r_B=10%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ECLG to GACL (ablation) | GACL · CIFAR-100 Si-Blurry, r_B=10% | A_Last: 47.30 ± 2.61 -> 70.31 ± 0.06 | Helped |
| add ECLG to GACL (ablation) | GACL · ImageNet-R Si-Blurry, r_B=10% | A_Last: 41.67 ± 0.36 -> 42.22 ± 0.10 | Helped |
| add ECLG to GACL (ablation) | GACL · Tiny-ImageNet Si-Blurry, r_B=10% | A_Last: 60.13 ± 0.37 -> 62.68 ± 0.08 | Helped |
| set regularization γ from 0 to 100 (ablation) | GACL · CIFAR-100 Si-Blurry | A_AUC: 8.87 ± 4.96 -> 57.99 ± 2.46 | Helped |
| set regularization γ from 100 to 10000 (ablation) | GACL · CIFAR-100 Si-Blurry | A_AUC: 57.99 ± 2.46 -> 51.01 ± 3.04 | Hurt |
| increase replay memory from 500 to 2000 | MVP-R · CIFAR-100 Si-Blurry | A_AUC: +4.42% | Helped |
| use GACL instead of SLDA | CIFAR-100 Si-Blurry · Si-Blurry, memory size 0 | A_AUC: 53.00 ± 3.85 -> 57.99 ± 2.46 | Helped |
| use GACL instead of MVP-R with memory 2000 | CIFAR-100 Si-Blurry · Si-Blurry, memory size 2000 | A_Last: 64.30 ± 0.29 -> 70.31 ± 0.06 | Helped |
| use GACL instead of MVP-R with memory 2000 | ImageNet-R Si-Blurry · Si-Blurry, memory size 2000 | A_AUC: 47.16 ± 1.00 -> 41.68 ± 0.78 | Hurt |
| use GACL instead of MVP-R with memory 2000 | Tiny-ImageNet Si-Blurry · Si-Blurry, memory size 2000 | A_Avg: 62.41 ± 0.50 -> 69.32 ± 0.87 | Helped |
Surprising findings
- GACL initially lags behind other EFCIL methods on CIFAR-100 in the first tasks, attributed to the dataset's class-number ratio growing fast, while its accuracy rises over tasks.
- Replay-based MVP-R with memory 2000 still beats GACL on AUC and Avg for CIFAR-100 and ImageNet-R.
Limitations
- Performance depends on a well-trained frozen backbone, since backbone weights are not updated.
- Lower task-wise accuracy in early tasks on CIFAR-100 compared with gradient-based EFCIL methods.
- Sensitive to extreme regularization: gamma=10000 causes underfitting; gamma=0 yields singular matrices.
Experimental setup
Datasets: CIFAR-100 ImageNet-R Tiny-ImageNet ImageNet (611 classes for backbone pretraining)
Benchmarks: CIFAR-100 (Si-Blurry) ImageNet-R (Si-Blurry) Tiny-ImageNet (Si-Blurry)
Metrics: AUC accuracy average incremental accuracy last-task accuracy
Baselines: EWC++ ER RM MVP-R LwF L2P DualPrompt MVP SLDA
Builds on: ACIL GKEAL Si-Blurry DeiT
Closest papers by meaning
- Advancing Prompt-Based Methods for Replay-Independent General Continual Learning (ICLR 2025 · similarity 0.89)
- Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach (NeurIPS 2024 · similarity 0.89)
- AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projection (NeurIPS 2025 · similarity 0.88)
- Class Incremental Learning via Likelihood Ratio Based Task Prediction (ICLR 2024 · similarity 0.87)
- F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning (NeurIPS 2024 · similarity 0.87)
- Non-exemplar Online Class-Incremental Continual Learning via Dual-Prototype Self-Augment and Refinement (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.