Research map / Continual learning / Class-incremental

Defying Imbalanced Forgetting in Class Incremental Learning

AAAI 2024 · Continual learning › Class-incremental · approach: Exemplar-efficient class-incremental learning · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Reveals imbalanced forgetting among old classes in replay-based class incremental learning and proposes CLAD, a plug-in regularizer that predicts vulnerable old classes via logits similarity and pushes new-class representations away from them.

Problem

Average incremental accuracy hides large accuracy imbalance among classes within the same old task in replay-based class incremental learning; forgetting is uneven across old classes because of representation conflicts with semantically similar new classes.

Approach

Predict which old classes are likely to be forgotten using the logits similarity between old-model outputs and new-class samples (Forgetting Prediction), then add a cosine-similarity regularizer that disentangles new-class representations from those conflict old classes, both online (in-batch) and offline (buffer), as a plugin to existing replay methods.

Key results

• CLAD improves LUCIR on CIFAR-100 (B=50, S=10) from 66.16% to 67.57% (+1.41), with gains up to +1.72 across settings.
• CLAD improves PODNet and CwD with smaller gains, e.g., CwD +1.81 on CIFAR-100 with S=5.
• Pearson correlation of 0.6 (p=0.00) between inter-class similarity and class-level forgetting; gains increase as exemplars per class decrease (R=5: +2.57 for LUCIR, S=10).

Measured effects reported in the paper

Surprising findings

• Random and smallest-similarity conflict class selection hurt performance, indicating the gain depends on targeting genuinely similar old classes.
• Using an oracle jointly-trained model for similarity prediction slightly decreases accuracy versus logits-based prediction from the first-task model.

Limitations

• Only one loss form and one old-class selection method were explored, as acknowledged by the authors.
• Exemplar-free CIL methods are not covered.
• Gains on PODNet are limited because its middle-layer feature distillation conflicts with the final-layer disentanglement.

Experimental setup

Datasets: CIFAR-100 ImageNet ImageNet-100

Benchmarks: CIFAR-100 ImageNet-100 ImageNet

Metrics: average incremental accuracy

Baselines: LwF iCaRL BiC LUCIR PODNet CwD CSCCT AANet

Builds on: LUCIR PODNet CwD iCaRL BiC

Closest papers by meaning

• Gradient Reweighting: Towards Imbalanced Class-Incremental Learning (CVPR 2024 · similarity 0.91)
• Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning (AAAI 2025 · similarity 0.91)
• Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.90)
• Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings (NeurIPS 2024 · similarity 0.89)
• Class Incremental Learning with Multi-Teacher Distillation (CVPR 2024 · similarity 0.88)
• Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting (ICLR 2025 · similarity 0.88)