Research map / Continual learning / Class-incremental
Reducing Class-wise Confusion for Incremental Learning with Disentangled Manifolds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CREATE, a class-incremental learning method that uses a lightweight per-class autoencoder classifier (reconstruction-error based) plus a confusion-aware latent separation loss to reduce class-wise confusion.
Problem
Prototype-based class incremental learning suffers from inadequate representation capacity (one prototype per class cannot model multi-modal class manifolds) and feature overlap caused by feature drift, leading to class-wise confusion and forgetting.
Approach
Replace prototype matching with a set of lightweight class-specific autoencoders whose reconstruction errors serve as classification scores, trained with cross-entropy, logit distillation, and a confusion-weighted contrastive loss in each class's latent subspace to separate overlapping features.
Key results
- On CIFAR100 Base50 Inc10, CREATE reaches 68.40% last and 75.52% average accuracy, versus 65.83% last for DSGD and 65.24% for BEEF on Base0 Inc20 last accuracy.
- On ImageNet100 Base50 Inc10, CREATE improves last accuracy by 2.44% over BEEF and average accuracy by 1.91%, using 14.44M parameters versus about 111.7M for DER/DSGD/BEEF.
- Ablation on CIFAR100 Base50 Inc10 shows average accuracy rising from 70.26% (NME) to 74.19% (AEs only) to 75.52% (full with LCR).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CREATE instead of Foster | CIFAR100 Base0 Inc10 · CIFAR100 Base0 Inc10 | average accuracy: 72.81 -> 75.60 | Helped |
| Use CREATE instead of DSGD | CIFAR100 Base0 Inc20 · CIFAR100 Base0 Inc20 | average accuracy: 72.91 -> 78.46 | Helped |
| Use CREATE instead of BEEF | CIFAR100 Base50 Inc5 · CIFAR100 Base50 Inc5 | average accuracy: 70.71 -> 72.27 | Helped |
| Use CREATE instead of BEEF | CIFAR100 Base50 Inc10 · CIFAR100 Base50 Inc10 | average accuracy: 71.70 -> 75.52 | Helped |
| Use CREATE instead of BEEF | ImageNet100 Base0 Inc10 · ImageNet100 Base0 Inc10 | last accuracy: 68.32 -> 66.94 | Hurt |
| Use CREATE instead of DER | ImageNet100 Base0 Inc20 · ImageNet100 Base0 Inc20 | average accuracy: 78.56 -> 81.37 | Helped |
| Use CREATE instead of BEEF | ImageNet100 Base50 Inc5 · ImageNet100 Base50 Inc5 | average accuracy: 77.69 -> 79.44 | Helped |
| Use CREATE instead of BEEF | ImageNet100 Base50 Inc10 · ImageNet100 Base50 Inc10 | average accuracy: 80.52 -> 82.43 | Helped |
| Add class-specific auto-encoders instead of NME (ablation) | CREATE · CIFAR100 Base50 Inc10 | average accuracy: 70.26 -> 74.19 | Helped |
| Add confusion-aware latent space separation loss (LCR) (ablation) | CREATE with class-specific auto-encoders · CIFAR100 Base50 Inc10 | average accuracy: 74.19 -> 75.52 | Helped |
| Increase AE channel size from 32 to 64 (ablation) | CREATE · CIFAR100 Base50 Inc10 | average accuracy: 75.52 -> 75.59 | No clear effect |
| Use two AE convolutional layers instead of one (ablation) | CREATE · CIFAR100 Base50 Inc10 | average accuracy: 75.52 -> 73.48 | Hurt |
Limitations
- Gains on some settings are small (e.g., +0.05% last accuracy on CIFAR100 Base0 Inc5 over DSGD)
- Evaluated only with ResNet18 trained from scratch on CIFAR100 and ImageNet100
Experimental setup
Datasets: CIFAR100 ImageNet100
Benchmarks: CIFAR100 ImageNet100
Metrics: last-phase accuracy average incremental accuracy parameter count (#P)
Baselines: iCaRL PODNet WA DER FOSTER DyTox BEEF DGR DSGD
Builds on: PyCIL iCaRL DER DSGD
Closest papers by meaning
- Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning (CVPR 2022 · similarity 0.88)
- Class-Incremental Learning With Strong Pre-Trained Models (CVPR 2022 · similarity 0.87)
- Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning (CVPR 2024 · similarity 0.86)
- Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning (ICML 2025 · similarity 0.86)
- Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.86)
- Compress to One Point: Neural Collapse for Pre-Trained Model-Based Class-Incremental Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.