Research map / Continual learning / Class-incremental
NICE: Neurogenesis Inspired Contextual Encoding for Replay-free Class Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NICE, a replay-free architectural continual learning method that groups neurons by maturation age and uses a learned activation-based context detector to select subnetworks at test time for class-incremental learning.
Problem
Class-incremental learning without replay or task identifiers: architectural CL methods rely on task IDs and underperform replay methods in CIL, causing catastrophic forgetting.
Approach
Inspired by adult hippocampal neurogenesis, neurons are assigned ages per episode, older neurons are frozen and pruned from younger ones to prevent interference, and a context detector (chained logistic regressions over binary thresholded activations) infers which episode's age-subnetwork to use at test time without task identifiers.
Key results
- NICE reaches 55.1% on CIFAR10 and 83.4% on MNIST at standard memory budget, exceeding iCaRL (54.2%) and other replay baselines on most datasets.
- With doubled memory (x2), NICE achieves 85.9 on MNIST, 75.1 on FashionMNIST, 69.4 on EMNIST, 56.5 on CIFAR10, outperforming replay baselines except on CIFAR100 and Tiny ImageNet.
- At quadrupled memory (x4), replay methods such as iCaRL and TAMiL begin to outperform NICE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NICE instead of TAMiL | Replay-free class-incremental learning · MNIST | accuracy: 81.1 -> 83.4 | Helped |
| Use NICE instead of iCaRL | Replay-free class-incremental learning · FashionMNIST | accuracy: 68.7 -> 73.9 | Helped |
| Use NICE instead of iCaRL | Replay-free class-incremental learning · EMNIST | accuracy: 65.3 -> 66.3 | Helped |
| Use NICE instead of iCaRL | Replay-free class-incremental learning · CIFAR10 | accuracy: 54.2 -> 55.1 | Helped |
| Use NICE instead of x-DER | Replay-free class-incremental learning · CIFAR100 | accuracy: 24.7 -> 20.3 | Hurt |
| Use NICE instead of x-DER | Replay-free class-incremental learning · Tiny ImageNet | accuracy: 11.6 -> 11.8 | Helped |
| Use only one VGG11 layer for context detection instead of all layers (ablation) | NICE · CIFAR10 and CIFAR100 | context detection accuracy | Hurt |
Surprising findings
- iCaRL, an early replay method, is among the top-3 performers across datasets, outperforming more recent baselines like x-DER and TAMiL on some datasets such as CIFAR10.
- Neurons in the network's early layers become frozen and depleted quickly, while later layers remain available for new classes.
Limitations
- NICE eventually exhausts available neurons as more classes are learned.
- Chaining logistic regressions is a simplistic model of activation-to-episode relationships, and variable-sized observations complicate fitting.
- Underperforms replay methods when memory budget is quadrupled (x4).
Experimental setup
Datasets: MNIST FashionMNIST EMNIST CIFAR10 CIFAR100 Tiny ImageNet
Benchmarks: MNIST FashionMNIST EMNIST CIFAR10 CIFAR100 Tiny ImageNet
Metrics: average accuracy across all classes after last episode forgetting
Baselines: iCaRL TAMiL x-DER DER GDumb ER FDR A-GEM SpaceNet AFAF
Builds on: Experience Replay DER iCaRL ReAct
Closest papers by meaning
- Dynamic Neural Regeneration: Enhancing Deep Learning Generalization on Small Datasets (NeurIPS 2024 · similarity 0.86)
- Task-Free Continual Generation and Representation Learning via Dynamic Expansionable Memory Cluster (AAAI 2024 · similarity 0.85)
- InTAct: Interval-based Task Activation Consolidation for Continual Learning (NeurIPS 2026 · similarity 0.84)
- Stay Hungry, Keep Learning: Sustainable Plasticity for Deep Reinforcement Learning (ICML 2025 · similarity 0.84)
- Rapid Learning without Catastrophic Forgetting in the Morris Water Maze (ICML 2024 · similarity 0.84)
- Non-exemplar Online Class-Incremental Continual Learning via Dual-Prototype Self-Augment and Refinement (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.