Research map / Continual learning / Class-incremental
Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPCR, an exemplar-free class-incremental learning method that estimates semantic shift with a dual projection (task-wise learnable linear projection plus category row-space projection) and reconstructs the classifier via BP-free ridge regression using calibrated covariances and prototypes.
Problem
Exemplar-free class-incremental learning suffers from catastrophic forgetting due to semantic shift of old-class embeddings after backbone updates and decision bias of classifiers trained only on new data.
Approach
Estimate semantic shift with a closed-form dual projection (task-wise linear projector combined with per-class row-space projection of uncentered covariance), calibrate stored class covariances and prototypes with it, and rebuild the classifier via ridge regression instead of backpropagation on new data only.
Key results
- On CIFAR-100 with T=10, DPCR reaches 50.24% final accuracy and 63.21% average incremental accuracy, versus 46.60% and 61.67% for the second-best LDC.
- On ImageNet-100 with T=10, DPCR achieves 52.16% final accuracy, improving over LDC by 2.18 points.
- Ablation on CIFAR-100 (T=10) shows final accuracy rising from 32.17% (RRCR only) to 51.04% with TSSP, CIP, and CN combined.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DPCR instead of the strongest listed baseline | EFCIL with ResNet-18 · CIFAR-100, T=10 | final accuracy: 46.80 -> 50.24 | Helped |
| Use DPCR instead of the strongest listed baseline | EFCIL with ResNet-18 · CIFAR-100, T=20 | final accuracy: 36.76 -> 38.98 | Helped |
| Use DPCR instead of the strongest listed baseline | EFCIL with ResNet-18 · Tiny-ImageNet, T=10 | final accuracy: 33.74 -> 35.20 | Helped |
| Use DPCR instead of the strongest listed baseline | EFCIL with ResNet-18 · Tiny-ImageNet, T=20 | final accuracy: 24.49 -> 26.54 | Helped |
| Use DPCR instead of the strongest listed baseline | EFCIL with ResNet-18 · ImageNet-100, T=10 | final accuracy: 49.98 -> 52.16 | Helped |
| Use DPCR instead of the strongest listed baseline | EFCIL with ResNet-18 · ImageNet-100, T=20 | final accuracy: 34.87 -> 38.35 | Helped |
| Use DPCR instead of the strongest listed baseline | EFCIL with ResNet-18 · CUB200, T=5 | final accuracy: 28.84 -> 29.51 | Helped |
| Use DPCR instead of the strongest listed baseline | EFCIL with ResNet-18 · ImageNet-1k, T=10 | final accuracy: 35.15 -> 35.49 | Helped |
| Add TSSP to RRCR (ablation) | RRCR on CIFAR-100, T=10 · CIFAR-100, T=10 | final accuracy: 32.17 -> 40.86 | Helped |
| Add CIP to RRCR+TSSP (ablation) | RRCR+TSSP on CIFAR-100, T=10 · CIFAR-100, T=10 | final accuracy: 40.86 -> 45.56 | Helped |
| Add CN to RRCR+TSSP+CIP (ablation) | RRCR+TSSP+CIP on CIFAR-100, T=10 · CIFAR-100, T=10 | final accuracy: 45.56 -> 51.04 | Helped |
Surprising findings
- Frozen-backbone analytic methods (ACIL, DS-AL, FeCAM) perform poorly relative to simpler baselines such as LwF and SDC, indicating the limits of freezing the backbone.
Limitations
- Relies on knowledge distillation (LwF-style) for backbone training, so representation drift is only partially constrained.
- Category-wise normalization is needed to handle numerical imbalance from non-unitary projections; regularization factor gamma must be tuned per dataset.
Experimental setup
Datasets: CIFAR-100 Tiny-ImageNet ImageNet-100 CUB200 ImageNet-1k
Benchmarks: CIFAR-100 Tiny-ImageNet ImageNet-100 CUB200 ImageNet-1k
Metrics: final accuracy average incremental accuracy
Baselines: LwF SDC PASS ACIL FeCAM DS-AL ADC LDC iCaRL PODNet FOSTER
Builds on: LwF LDC ACIL ADC SDC
Closest papers by meaning
- Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning (NeurIPS 2024 · similarity 0.88)
- Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning (ICML 2025 · similarity 0.87)
- Elastic Feature Consolidation For Cold Start Exemplar-Free Incremental Learning (ICLR 2024 · similarity 0.85)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.85)
- Gradient Reweighting: Towards Imbalanced Class-Incremental Learning (CVPR 2024 · similarity 0.85)
- Dynamic Integration of Task-Specific Adapters for Class Incremental Learning (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.