Research map / Continual learning / Class-incremental
Prediction Error-based Classification for Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Prediction Error-based Classification (PEC), which scores each class by the error of a class-specific student trained to mimic a frozen random teacher, and applies it to single-pass class-incremental learning.
Problem
Class-incremental learning suffers from forgetting and score imbalance across classes never seen together; existing discriminative and generative classifiers struggle, especially in single-pass or one-class-per-task streams.
Approach
Replace discriminative logits or generative likelihoods with per-class student networks trained to replicate a frozen random teacher; the class whose student has the smallest prediction error wins. Each class has its own module, so there is no interference, and the method is motivated as an approximation of a Gaussian Process posterior-variance rule.
Key results
- PEC reaches 92.31% on MNIST (10/1) and 58.94% on CIFAR-10 (10/1), the highest among all methods in those settings.
- PEC outperforms all rehearsal-free baselines in every considered setting, and rehearsal-based methods with a 500-example buffer in most cases.
- PEC achieves 14.90% on miniImageNet (100/1) and 26.54% on CIFAR-100 (100/1), above the best rehearsal-based results there.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PEC instead of rehearsal-free baselines | Class-incremental learning · All reported benchmarks and task splits | final average accuracy | Helped |
| Use PEC instead of rehearsal-based methods with a 500-example buffer | Class-incremental learning · Most reported benchmarks | final average accuracy | Mixed |
| Equalize training iterations across classes (ablation) | PEC · Imbalanced CIFAR-10 and SVHN | final average accuracy (%): 28.96 ($\pm$ 0.21) -> 55.84 ($\pm$ 0.14); 21.73 ($\pm$ 0.14) -> 68.65 ($\pm$ 0.21) | Helped |
| Rescale class scores using oracle balancing (ablation) | PEC · CIFAR-10 | final average accuracy (%): 58.94 ($\pm$ 0.09) -> 61.01 ($\pm$ 0.11) | Helped |
| Rescale class scores using buffer balancing (ablation) | PEC · CIFAR-10 | final average accuracy (%): 58.94 ($\pm$ 0.09) -> 59.02 ($\pm$ 0.25) | No clear effect |
| Use PEC on imbalanced rather than balanced data (ablation) | PEC · CIFAR-10 | final average accuracy (%): 58.94 ($\pm$ 0.09) -> 28.96 ($\pm$ 0.21) | Hurt |
| Use fixed learning rate 0.001 instead of tuned learning rate (ablation) | PEC · MNIST | final average accuracy (%): 92.31 ($\pm$ 0.13) -> 90.51 ($\pm$ 0.16) | Hurt |
| Use fixed learning rate 0.001 instead of tuned learning rate (ablation) | PEC · CIFAR-100 and miniImageNet | final average accuracy (%): 26.54 ($\pm$ 0.11) -> 26.54 ($\pm$ 0.11); 14.90 ($\pm$ 0.08) -> 14.90 ($\pm$ 0.08) | No clear effect |
| Increase network depth (ablation) | PEC · MNIST and CIFAR-10 | final average accuracy | No clear effect |
| Increase network width or output dimension (ablation) | PEC · MNIST and CIFAR-10 | final average accuracy | Helped |
| Use Kaiming initialization rather than alternative initialization schemes (ablation) | PEC student and teacher networks · PEC benchmarks | final average accuracy | No clear effect |
| Use PEC rather than ER and VAE-GC while varying epochs and model parameters | Class-incremental learning · Evaluated regimes | final average accuracy | Helped |
Surprising findings
- Score imbalance from unequal per-class dataset sizes severely degrades PEC (CIFAR-10 58.94% drops to 28.96% when imbalanced), and equal-budget training largely fixes it.
- Using the default learning rate 0.001 without tuning still keeps PEC ahead of tuned rehearsal-free baselines.
Limitations
- Complete separation of class modules prevents any transfer of information between classes.
- Parameter count grows linearly with the number of classes, one student per class.
- Performance drops substantially under class-imbalanced training data unless balancing strategies are used.
Experimental setup
Datasets: MNIST SVHN CIFAR-10 CIFAR-100 miniImageNet
Benchmarks: MNIST (10/1) Balanced SVHN (10/1) CIFAR-10 (10/1) CIFAR-100 (100/1) miniImageNet (100/1) MNIST (5/2) CIFAR-10 (5/2) CIFAR-100 (10/10) miniImageNet (20/5)
Metrics: final average accuracy
Baselines: ER A-GEM DER DER++ X-DER ER-ACE iCaRL BiC EWC + LT SI + LT LwF + LT Nearest mean
Builds on: Random Network Distillation Experience Replay VAE-GC SLDA Labels trick
Closest papers by meaning
- AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projection (NeurIPS 2025 · similarity 0.86)
- Continual Learning Using a Kernel-Based Method Over Foundation Models (AAAI 2025 · similarity 0.86)
- Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings (NeurIPS 2024 · similarity 0.86)
- Mixture of Noise for Pre-Trained Model-Based Class-Incremental Learning (NeurIPS 2025 · similarity 0.86)
- Class Incremental Learning via Likelihood Ratio Based Task Prediction (ICLR 2024 · similarity 0.86)
- Incremental Sequence Classification with Temporal Consistency (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.