Research map / Continual learning / Class-incremental
DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DS-AL, an exemplar-free class-incremental learning method that pairs a closed-form recursive least-squares main stream (C-RLS) with a dual-activation compensation stream that fits the main stream's null space.
Problem
Exemplar-free class-incremental learning (EFCIL) suffers catastrophic forgetting and, for analytic-learning (AL) based methods, under-fitting from relying on a single linear projection.
Approach
Recasts CIL as a concatenated recursive least squares problem that is equivalent to joint training on a frozen backbone, and adds a compensation stream that re-activates embeddings with a different activation (Tanh) and fits the residue in the main stream's null space, with previous-label cleansing to avoid false supervision on old classes.
Key results
- On ImageNet-Full with K=5, DS-AL reaches 67.18% average accuracy and 58.17% last-phase accuracy, versus 65.34% and 56.11% for ACIL.
- DS-AL outperforms replay-based methods from K>=25 on ImageNet-Full, e.g., 66.81% vs 63.93% average accuracy for RMM at K=25.
- DS-AL achieves 66.39% average accuracy on CIFAR-100 at K=5 (ResNet-32), slightly above ACIL's 66.30%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Dual-Activation Compensation (DAC) to C-RLS (ablation) | DS-AL · ImageNet-Full, K=5 | last-phase accuracy: 56.11 -> 57.43 | Helped |
| add Previous Label Cleansing (PLC) to C-RLS + DAC (ablation) | DS-AL · ImageNet-Full, K=5 | last-phase accuracy: 57.43 -> 58.17 | Helped |
| add DAC to the main stream (ablation) | DS-AL · ImageNet-Full, K=5 | average accuracy: 65.34 -> 66.71 | Helped |
| add PLC to C-RLS + DAC (ablation) | DS-AL · ImageNet-Full, K=5 | average accuracy: 66.71 -> 67.18 | Helped |
| increase compensation ratio beyond the task-appropriate level (ablation) | DS-AL · CIFAR-100, K=5 | last-phase accuracy | Hurt |
| use Tanh rather than other tested compensation activations (ablation) | DS-AL · CIFAR-100, ImageNet-100, and ImageNet-Full, K=5 | last-phase accuracy | Mixed |
Surprising findings
- Replay-based methods outperform DS-AL at small phase counts (e.g., K=5 on ImageNet-Full), with the gap reversing as K increases.
Limitations
- Compensation ratio and activation function are hyperparameters chosen by grid search on the evaluation settings.
- Performance gap to replay-based methods remains at small phase counts such as K=5.
- Backbone is frozen after BP-based base training, so features do not adapt to incremental data.
Experimental setup
Datasets: CIFAR-100 ImageNet-100 ImageNet-Full
Benchmarks: CIFAR-100 ImageNet-100 ImageNet-Full
Metrics: average incremental accuracy last-phase accuracy
Baselines: LwF PASS IL2A ACIL FeTrIL iVoro-ND LUCIR PODNet AANets RMM FOSTER
Builds on: ACIL RLS PLC
Closest papers by meaning
- Tensor Decomposition Based Memory-Efficient Incremental Learning (ICML 2025 · similarity 0.87)
- Non-exemplar Online Class-Incremental Continual Learning via Dual-Prototype Self-Augment and Refinement (AAAI 2024 · similarity 0.86)
- Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models (NeurIPS 2024 · similarity 0.86)
- Salient Frequency-aware Exemplar Compression for Resource-constrained Online Continual Learning (AAAI 2025 · similarity 0.86)
- F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning (NeurIPS 2024 · similarity 0.86)
- GACL: Exemplar-Free Generalized Analytic Continual Learning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.