Research map / Architectures and efficiency / Pruning and efficient tuning
Learning Without Forgetting
Zhizhong Li, Derek Hoiem
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LwF response-preserving loss instead of fine-tuning | AlexNet ImageNet->CUB · ImageNet->CUB (val) | old-task accuracy: -3.8 vs LwF 54.7 | Improved |
| add LwF instead of feature extraction | AlexNet ImageNet->CUB · ImageNet->CUB (val) | new-task accuracy: feature extraction -5.2 vs LwF 57.7 | Improved |
| make more layers task-specific (last hidden layer) (ablation) | LwF AlexNet · ImageNet->CUB (val) | new-task accuracy: 57.7 -> 56.2 | No clear effect |
| add network expansion with frozen original weights (ablation) | AlexNet · ImageNet->CUB (val) | new-task accuracy: 57.7 -> 54.0 | Hurt |
| add LwF to network expansion (ablation) | network expansion · ImageNet->CUB (val) | new-task accuracy: 54.0 -> 57.0 | Improved |
| lower shared-parameter learning rate to 10% in fine-tuning (ablation) | fine-tuning AlexNet · ImageNet->CUB (val) | old-task accuracy: 52.2 vs LwF 54.7 | No clear effect |
| add warm-up step to fine-tuning (ablation) | fine-tuning AlexNet · ImageNet->CUB (val) | old-task accuracy: 42.5 -> 50.9 | Improved |
| add warm-up step to LwF (ablation) | LwF AlexNet · ImageNet->CUB (val) | old-task accuracy: 53.5 -> 54.7 | No clear effect |
| regularize outputs (LwF) instead of L2 on parameters (ablation) | fine-tuning · ImageNet->Scenes / Places365->VOC | accuracy | Improved |
| use knowledge distillation loss instead of L1, L2 or cross-entropy (ablation) | LwF · old/new task accuracy | accuracy | No clear effect |
| LwF versus joint training on AlexNet | LwF · ImageNet->CUB (val) | old-task accuracy: 0.6 vs LwF 54.7 (joint minus LwF) | Mixed |
| replace fine-tuning with LwF in MD-Net online training | MD-Net · VOT 2015 | expected average overlap: 0.373 -> 0.383 | No clear effect |
Closest papers by meaning
- Forgetting Knowledge Localization and Isolation for Continual Forgetting of Pre-trained Vision Models (AAAI 2026 · similarity 0.88)
- Sequential Mastery of Multiple Visual Tasks: Networks Naturally Learn to Learn and Forget to Forget (CVPR 2020 · similarity 0.88)
- A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning (TPAMI 2024 · similarity 0.88)
- A continual learning survey: Defying forgetting in classification tasks (TPAMI 2021 · similarity 0.88)
- Mixed-Privacy Forgetting in Deep Networks (CVPR 2021 · similarity 0.88)
- REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
