Research map / Continual learning / Class-incremental
Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning
Nader Asadi, MohammadReza Davari, Sudhir Mudur, Rahaf Aljundi, Eugene Belilovsky
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add prototype-sample relation distillation loss (beta>0) vs none (beta=0) (ablation) | PRD · Split-CIFAR100 K=20 task-incremental | average observed accuracy: 39.4 -> 83.5 at beta=4 | Improved |
| add prototype-sample relation distillation loss (beta>0) vs none (beta=0) (ablation) | PRD · Split-MiniImageNet K=20 task-incremental | average observed accuracy: 31.2 -> 70.9 at beta=8 | Improved |
| add prototype-sample relation distillation loss (beta>0) vs none (beta=0) (ablation) | PRD · ImageNet32 K=200 task-incremental | average observed accuracy: 21.3 -> 67.5 at beta=16 | Improved |
| increase relation distillation coefficient beta (ablation) | PRD · Split-CIFAR100 K=20 task-incremental | average observed accuracy: 80.0 to 83.5 across beta=1 to 4, 82.7 at 16 | No clear effect |
| add prototype learning loss (alpha>0) vs none (alpha=0) (ablation) | PRD · Split-CIFAR100 K=20 task-incremental | average observed accuracy: 20.6 -> 83.5 at alpha=4 | Improved |
| add prototype learning loss (alpha>0) vs none (alpha=0) (ablation) | PRD · Split-MiniImageNet K=20 task-incremental | average observed accuracy: 20.2 -> 69.8 at alpha=2 | Improved |
| vary prototype coefficient alpha in {1,2,4,8,16} (ablation) | PRD · Split-CIFAR100 and Split-MiniImageNet K=20 task-incremental | average observed accuracy: 81.8-83.5 (CIFAR100), 67.4-69.8 (MiniImageNet) for alpha>=1 | No clear effect |
| use PRD (replay-free) instead of replay-based ER with 50 samples per class | Split-CIFAR100 K=20 task-incremental | average observed accuracy: PRD outperforms ER M=50 without stored data | Improved |
| add a small replay buffer (M=5/20/50) to PRD | PRD · Split-CIFAR100 K=20 class-incremental | average observed accuracy: 27.8 (M=0) -> 32.0 (M=5) -> 39.5 (M=20) -> 45.1 (M=50) | Improved |
| replay-free PRD vs replay-based methods in class-incremental | PRD · Split-CIFAR100 K=20 class-incremental | average observed accuracy: 27.8 (M=0) vs ER M=5 15.2 | Improved |
| replay-free PRD vs SPB and distillation baselines with pre-trained initialization | PRD · Split-CIFAR100 and ImageNet-Subset pre-trained init class-incremental | average cumulative incremental accuracy: PRD 64.3 vs SPB 60.9 on Split-CIFAR100 K=6; 71.8 vs 68.7 on ImageNet-Subset K=6 | Improved |
| replay-free PRD vs EWC and LwF in domain-incremental CLAD-C | PRD · CLAD-C domain-incremental | AMCA: 65.1 vs LwF 63.7 and EWC 62.5 | Improved |
Closest papers by meaning
- Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillation (AAAI 2024 · similarity 0.92)
- Non-exemplar Online Class-Incremental Continual Learning via Dual-Prototype Self-Augment and Refinement (AAAI 2024 · similarity 0.90)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.89)
- Continual Learning with Global Alignment (NeurIPS 2024 · similarity 0.89)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.89)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
