Research map / Continual learning / Class-incremental
Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a mathematical framework separating task confusion from catastrophic forgetting in class-incremental learning, proving discriminative modeling cannot reach optimal class-IL while generative modeling can, and assesses popular class-IL strategies against it.
Problem
Class-incremental learning must classify all seen classes without task-IDs; performance drops are usually attributed to catastrophic forgetting, but task confusion (inter-task discrimination never trained) lacks theoretical understanding.
Approach
Decomposes an N-way classifier loss into pairwise binary-classifier loss blocks arranged in a task-level matrix, defines CF (diagonal blocks) and TC (off-diagonal blocks) separately, and proves an Infeasibility Theorem for discriminative modeling and a Feasibility Theorem for generative modeling.
Key results
- None and regularization schemes (EWC, SI) in class-IL stay near the 100/T% bound, e.g., None 18.74% on CIFAR-10 with T=5 and 7.96% on CIFAR-100 with T=10.
- Generative classifiers reach higher class-IL accuracy than discriminative methods, e.g., GenC 56.02% on CIFAR-10 and 49.53% on CIFAR-100 versus AR1 25.13% and 21.01%.
- Generative replay DGR reaches 91.12% on MNIST but only 18.13% on CIFAR-10, indicating surrogate densities fail on larger datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a generative classifier (GenC) instead of None | Class-IL · MNIST | accuracy: 19.92 ± 0.02 -> 93.75 ± 0.09 | Helped |
| Use a generative classifier (GenC) instead of None | Class-IL · CIFAR-10 | accuracy: 18.74 ± 0.29 -> 56.02 ± 0.04 | Helped |
| Use a generative classifier (GenC) instead of None | Class-IL · CIFAR-100 | accuracy: 7.96 ± 0.11 -> 49.53 ± 0.07 | Helped |
| Use a generative classifier (GenC) instead of None | Class-IL · CORe50 | accuracy: 18.65 ± 0.26 -> 70.80 ± 0.10 | Helped |
| Add regularization (EWC) to discriminative class-IL | None · CIFAR-10 | accuracy: 18.74 ± 0.29 -> 18.77 ± 0.31 | No clear effect |
| Add bias correction (AR1) to discriminative class-IL | None · CIFAR-10 | accuracy: 18.74 ± 0.29 -> 25.13 ± 1.18 | Helped |
| Use generative replay (DGR) instead of None | Class-IL · CIFAR-10 | accuracy: 18.74 ± 0.29 -> 18.13 ± 1.85 | No clear effect |
| Use generative replay (DGR) instead of None | Class-IL · MNIST | accuracy: 19.92 ± 0.02 -> 91.12 ± 0.65 | Helped |
| Use generative classifiers (SLDA and GenC) rather than discriminative class-IL strategies | Class-IL · CIFAR-10 | accuracy: None: 18.74 ± 0.29; SLDA: 38.33 ± 0.04; GenC: 56.02 ± 0.04 | Helped |
| Use generative classifiers (SLDA and GenC) rather than discriminative class-IL strategies | Class-IL · CIFAR-100 | accuracy: None: 7.96 ± 0.11; SLDA: 44.49 ± 0.00; GenC: 49.53 ± 0.07 | Helped |
| Use generative replay rather than a generative classifier | Class-IL · CIFAR-100 | accuracy: DGR: 9.41 ± 0.30; BI-R+SI: 34.34 ± 0.23; GenC: 49.53 ± 0.07 | Mixed |
| Use generative modeling instead of discriminative modeling | Class-IL · Task-confusion performance | classification accuracy | Helped |
Surprising findings
- Some entropy-augmented regularization results reported in prior work exceed the 100/T% upper bound, which the authors note as an unresolved discrepancy.
- SI substantially mitigates catastrophic forgetting yet has no effect on task confusion.
Limitations
- Theorems rely on an incompatibility assumption between task objectives that is argued informally rather than verified empirically.
- Experiments use a CNN on standard benchmarks with results reported as means over 10 runs; the paper notes some empirical results contradict the 100/T% bound without resolving it.
Experimental setup
Datasets: MNIST CIFAR-10 CIFAR-100 CORe50
Benchmarks: MNIST CIFAR-10 CIFAR-100 CORe50
Metrics: classification accuracy
Baselines: EWC SI LwF CWR AR1 DGR BI-R SLDA GenC
Builds on: Elastic Weight Consolidation Synaptic Intelligence Learning without Forgetting Deep Generative Replay
Closest papers by meaning
- Generative Multi-modal Models are Good Class Incremental Learners (CVPR 2024 · similarity 0.89)
- Defying Imbalanced Forgetting in Class Incremental Learning (AAAI 2024 · similarity 0.89)
- Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning (AAAI 2025 · similarity 0.88)
- Revisiting Generative Replay for Class Incremental Object Detection (CVPR 2025 · similarity 0.88)
- Continual Learning Using a Kernel-Based Method Over Foundation Models (AAAI 2025 · similarity 0.87)
- Class Incremental Learning via Likelihood Ratio Based Task Prediction (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.