Research map / Continual learning / Class-incremental
Harnessing Neural Unit Dynamics for Effective and Scalable Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoActivator, a rehearsal-free class-incremental learning model built from randomized, supervisory-gated neural units that grow minimally per task, with closed-form output-weight updates and activation-threshold-based task-agnostic reactivation.
Problem
Class-incremental learning must learn new classes from sequential data without forgetting old ones and without task identities at test time, while keeping model growth and memory budget small.
Approach
Grows neural units by randomly generating hidden node batches and recruiting only those satisfying a supervisory (residual-error-reduction) condition, updates the output weights in closed form via pseudoinverse stepwise updates, and at inference reactivates units by matching per-unit activation thresholds computed from predicted probabilities.
Key results
- On split MNIST-10/5 the method reaches 97.32% ACA with zero exemplar buffer, versus 95.51% for the best listed competitor (IL2M).
- On CIFAR-100/10 it reaches 69.65% ACA with a scale ratio of 114.26%, versus 67.54% for FS-DGPM.
- On ImageNet-R with pre-trained ViT, the DualPrompt-combined variant reaches 70.32% ACA, versus 69.01% for CODA-Prompt.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AutoActivator instead of IL2M | MLP on MNIST-10/5 · MNIST-10/5 | ACA (%): 94.87 -> 97.32 | Helped |
| Use AutoActivator instead of CRNet | MLP on FashionMNIST-10/5 · FashionMNIST-10/5 | ACA (%): 86.01 -> 88.46 | Helped |
| Use AutoActivator instead of FS-DGPM | ResNet-18 on CIFAR-100/10 · CIFAR-100/10 | ACA (%): 67.54 -> 69.65 | Helped |
| Use AutoActivator instead of FS-DGPM | ResNet-18 on CIFAR-100/25 · CIFAR-100/25 | ACA (%): 68.45 -> 70.16 | Helped |
| Attach complementary prompts to two pre-trained ViTs | AutoActivator on ImageNet-R-200/10 · ImageNet-R-200/10 | ACA (%): 68.13 -> 70.32 | Helped |
| Add Component I for rectifying inter-class confusion (ablation) | AutoActivator reactivation · MNIST-10/5 | ACA (%): 13.69 -> 97.18 | Helped |
| Add Component II for calibrating reactivation results (ablation) | AutoActivator reactivation with Component I · FashionMNIST-10/5 | ACA (%): 86.46 -> 88.37 | Helped |
| Add Component II for calibrating reactivation results (ablation) | AutoActivator reactivation with Component I · ImageNet-R | ACA (%): 63.25 -> 65.83 | Helped |
| Use batch-by-batch node recruitment (l=10) instead of one-by-one recruitment (l=1) (ablation) | AutoActivator node expansion · CIL task sequences | training time | Helped |
| Use one-by-one node recruitment (l=1) instead of batch-by-batch recruitment (l=10) (ablation) | AutoActivator node expansion · CIL task sequences | neural unit size | Helped |
| Increase the maximum random-generation attempts T_max (ablation) | AutoActivator candidate-node generation · CIL task sequences | ACA | Helped |
| Increase expected accuracy R(t) from 95% to 99% (ablation) | AutoActivator on FashionMNIST-10/5 · FashionMNIST-10/5 | parameters per class (M): 0.0110 -> 0.0535 | Hurt |
Surprising findings
- Pre-training does not necessarily translate into better CIL performance; some methods suffer large forgetting when given a pre-trained backbone compared with learning from scratch.
Limitations
- Assumes task boundaries are available during training, which the authors note is restrictive for real-world use.
- Relies on a pre-trained backbone assumption; fairness and bias from the pre-trained model may persist.
Experimental setup
Datasets: MNIST FashionMNIST CIFAR-100 ImageNet-R Tiny-ImageNet ImageNet ImageNet-100 ImageNet-1K
Benchmarks: split MNIST-10/5 split FashionMNIST-10/5 split CIFAR-100/10 split CIFAR-100/25 split ImageNet-R-200/10 CIFAR-100/100 single CIL ImageNet-100 ImageNet-1K
Metrics: ACA BWT memory budget AIA forgetting scale ratio
Baselines: IL2M BiC LOGD FS-DGPM Co2L ARI NISPA RPS-Net DER PCL L2P DualPrompt
Builds on: randomized learning (random hidden nodes) incremental pseudoinverse update BiC PCL
Closest papers by meaning
- Compress to One Point: Neural Collapse for Pre-Trained Model-Based Class-Incremental Learning (AAAI 2025 · similarity 0.87)
- Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning (CVPR 2024 · similarity 0.87)
- Knowledge Memorization and Rumination for Pre-trained Model-based Class-Incremental Learning (CVPR 2025 · similarity 0.87)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.86)
- eTag: Class-Incremental Learning via Embedding Distillation and Task-Oriented Generation (AAAI 2024 · similarity 0.86)
- MIND: Multi-Task Incremental Network Distillation (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.