Research map / Continual learning / Online and replay-based
Probing Representation Forgetting in Supervised and Unsupervised Continual Learning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate continual learning with an optimal linear probe on old-task data instead of the trained classifier | Naive finetuning · SplitCIFAR10, Task 1 after Task 2 | Task 1 accuracy degradation: observed accuracy decreases from $85\%$ to $63\%; LP accuracy degradation is only $5.7\% | Helped |
| Use SupCon rather than cross-entropy for finetuning | Continual finetuning · Long task sequences including SplitCIFAR100 and SplitMiniImageNet | LP accuracy on earlier tasks | Helped |
| Use SimCLR rather than supervised CE or LwF | Unsupervised continual representation learning · SplitCIFAR100 | LP accuracy retention on Task 1 | Mixed |
| Add NME-M5 evaluation-time class prototypes to SupCon finetuning | SupCon finetuning · SplitCIFAR100, 10 tasks | Observed Task 1 accuracy at T=10: $20.8\%$ -> $48.0\%$ | Helped |
| Increase model width from 32 to 128 (ablation) | Finetuning (CE), offline SplitCIFAR100 · SplitCIFAR100, 10 tasks | Task 1 LP accuracy at T=10: 64.8 -> 70.5 | Helped |
| Increase model depth from RN18 to RN101 at width 32 (ablation) | Finetuning (CE), offline SplitCIFAR100 · SplitCIFAR100, 10 tasks | Task 1 LP accuracy at T=10: 64.8 -> 67.9 | Helped |
| Increase model width from 32 to 128 (ablation) | Finetuning (CE), online SplitCIFAR100 · SplitCIFAR100, 10 tasks | Task 1 LP accuracy at T=10: 39.8 -> 42.3 | Helped |
| Increase model depth from RN18 to RN101 at width 32 (ablation) | Finetuning (CE), online SplitCIFAR100 · SplitCIFAR100, 10 tasks | Task 1 LP accuracy at T=10: 39.8 -> 28.2 | Hurt |
| Increase ER memory from M=5 to M=20 (ablation) | ER, offline SplitCIFAR100, RN18 width 32 · SplitCIFAR100, 10 tasks | Task 1 LP accuracy at T=10: 74.2 -> 76 | Helped |
| Increase EWC regularization strength from lambda 0.5k to 8k (ablation) | EWC · ImageNet to CUB | LP accuracy on ImageNet after task 2: $63.99\%$ -> $67.46\%$ | Helped |
| Increase EWC regularization strength from lambda 0.5k to 8k (ablation) | EWC · ImageNet to CUB | Current-task accuracy: $74.53\%$ -> $72.70\%$ | Hurt |
Closest papers by meaning
- Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (ICML 2025 · similarity 0.95)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.91)
- Do Your Best and Get Enough Rest for Continual Learning (CVPR 2025 · similarity 0.91)
- Continual Learning with Global Alignment (NeurIPS 2024 · similarity 0.91)
- Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers (ICML 2025 · similarity 0.90)
- Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.