Research map / Continual learning / Class-incremental
Fine-Grained Knowledge Selection and Restoration for Non-exemplar Class Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a ViT-based non-exemplar class-incremental learning framework with patch-level knowledge selection (distillation weighted by patch-to-CLS distance) and task-agnostic prototype restoration to reduce forgetting and classifier bias.
Problem
Non-exemplar class incremental learning (NECIL), where catastrophic forgetting must be mitigated without storing any past-task samples, and classifier bias arises from prototype replay.
Approach
Uses vision transformer patch embeddings to weight knowledge distillation per patch by its relevance to the [CLS] token (foreground patches get less distillation, background patches more), and restores old-class prototypes by adding current-sample offsets to old class centers, with an MSE regularizer making offsets task-agnostic.
Key results
- On CIFAR100 with 5 tasks, the method reaches 68.17% average accuracy versus 65.88% for SSRE, the best non-exemplar baseline in the table.
- Ablation on CIFAR100 (5 tasks): patch-level knowledge selection alone improves accuracy from 56.44% (ViT baseline) to 58.27%, and the full method reaches 59.02%.
- Applying prototype restoration to CNN-based PASS and SSRE raises CIFAR100 accuracy by roughly 2-3 points across task settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add patch-level knowledge selection (PKS) (ablation) | ViT baseline · CIFAR100, 5/10/20 tasks | top-1 accuracy: 56.44 -> 58.27; 51.90 -> 56.82; 51.20 -> 52.87 | Helped |
| Add prototype restoration (PR) (ablation) | ViT baseline · CIFAR100, 5/10/20 tasks | top-1 accuracy: 56.44 -> 57.78; 51.90 -> 54.61; 51.20 -> 51.51 | Helped |
| Add PKS and PR (ablation) | ViT baseline · CIFAR100, 5/10/20 tasks | top-1 accuracy: 56.44 -> 59.02; 51.90 -> 57.90; 51.20 -> 54.25 | Helped |
| Set all patch distillation weights to 1 instead of using the proposed patch weights (ablation) | PKS · CIFAR100, 5/10/20 tasks | top-1 accuracy: 56.18; 51.99; 50.53 vs 59.02; 57.90; 54.25 | Hurt |
| Use patch-to-[CLS] distance as the weight, rather than the inverse-distance rule (ablation) | PKS · CIFAR100, 5/10/20 tasks | top-1 accuracy: 53.81; 49.78; 48.31 vs 59.02; 57.90; 54.25 | Hurt |
| Add prototype restoration (PR) (ablation) | PASS · CIFAR100, 5/10/20 tasks | top-1 accuracy: 55.67 -> 57.45; 49.03 -> 52.30; 48.48 -> 51.22 | Helped |
| Add prototype restoration (PR) (ablation) | SSRE · CIFAR100, 5/10/20 tasks | top-1 accuracy: 56.33 -> 58.02; 55.01 -> 57.10; 50.47 -> 53.28 | Helped |
| Apply PKS and PR (ablation) | DyTox · CIFAR-100, 10/20/50 tasks | average accuracy: 75.47 -> 78.35; 75.10 -> 77.63; 73.89 -> 76.52 | Helped |
| Apply PKS and PR (ablation) | DyTox · CIFAR-100, 10/20/50 tasks | last accuracy: 62.10 -> 66.47; 59.41 -> 63.90; 57.21 -> 61.45 | Helped |
| Apply PKS and PR (ablation) | DyTox · CIFAR-100, 10/20/50 tasks | forgetting: 15.43 -> 13.12; 21.60 -> 15.76; 24.22 -> 20.23 | Helped |
| Use the full proposed method instead of SSRE | NECIL models · TinyImageNet, 5 tasks | average forgetting: 9.17 vs 11.45 | Hurt |
| Use the full proposed method instead of SSRE | NECIL models · CIFAR100, 5/10/20 tasks; TinyImageNet, 10/20 tasks; ImageNet-Subset, 10 tasks | average forgetting: 18.37 -> 16.36; 19.48 -> 17.16; 18.37 -> 16.61; 14.06 -> 12.21; 14.20 -> 12.82; 23.22 -> 18.39 | Helped |
Surprising findings
- Assigning uniform weights to all patches in distillation or weighting patches proportional to their distance from [CLS] both perform worse than the proposed relevance weighting, with the latter falling below the ViT baseline.
Limitations
- Relevance metric is a simple L2 distance to the [CLS] token, which the authors describe as one simple choice without losing generality.
- Prototype restoration relies on the assumption that per-sample offsets are task-agnostic, enforced only via an MSE regularizer.
Experimental setup
Datasets: CIFAR100 TinyImageNet ImageNet-Subset
Benchmarks: CIFAR100 TinyImageNet ImageNet-Subset
Metrics: average accuracy last accuracy average forgetting
Baselines: PASS SSRE IL2A EWC LwF-MC MUC iCaRL EEIL LUCIR DyTox
Builds on: PASS SSRE DyTox LwF
Closest papers by meaning
- Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning (NeurIPS 2026 · similarity 0.87)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.87)
- Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning (AAAI 2025 · similarity 0.87)
- Knowledge Memorization and Rumination for Pre-trained Model-based Class-Incremental Learning (CVPR 2025 · similarity 0.86)
- Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method (ICML 2024 · similarity 0.86)
- Knowledge Graph Enhanced Generative Multi-modal Models for Class-Incremental Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.