Research map / Continual learning / Online and replay-based
Enhancing Knowledge Transfer for Task Incremental Learning with Data-free Subnetwork
Qiang Gao, Xiaojun Shan, Yuchen Zhang, Fan Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace WSN weight-wise subnetwork with DSN | WSN · CIFAR-100 | ACC: 72.84 -> 75.17 | Improved |
| replace HAT with DSN | HAT · PMNIST | ACC: 97.10 -> 98.24 | Improved |
| replace WSN with DSN | WSN · TinyImageNet | ACC: 45.96 -> 46.56 | Improved |
| replace WSN with DSN | WSN · RMNIST | ACC: 97.32 -> 97.73 | Improved |
| add data-free replay and fine-tuning of similar task subnetwork (backward transfer) | forgetting-free baselines (HAT, WSN) · CIFAR-100 | BWT: 0.00 -> 0.02 | Improved |
| add neuron-wise masks and knowledge transfer from similar tasks | WSN · CIFAR-100 | Trans: -11.00 -> -8.67 | Improved |
| network expansion (PGN, RCL) versus fixed network with regularization (EWC) | PGN/RCL · CIFAR-100 | ACC: 67.15 -> 53.84 | Hurt |
| mean-IMM versus naive SGD | mean-IMM · PMNIST | ACC: 81.37 -> 80.10 | Hurt |
| larger over-parameterized hypernetwork (more neurons per layer) (ablation) | DSN · RMNIST/CIFAR-100/TinyImageNet | accuracy | Improved |
| smaller initial learning rate for hypernetwork (ablation) | DSN · RMNIST/CIFAR-100/TinyImageNet | accuracy | Improved |
| increase eta (neuron-count penalty) to reserve room for future tasks (ablation) | DSN · PMNIST/TinyImageNet | accuracy | Hurt |
| set eta = 0 (no penalty on activated neurons) (ablation) | DSN · PMNIST/TinyImageNet | accuracy and capacity | Hurt |
Closest papers by meaning
- MIND: Multi-Task Incremental Network Distillation (AAAI 2024 · similarity 0.88)
- Forget-free Continual Learning with Winning Subnetworks (ICML 2022 · similarity 0.87)
- Continual Learning: Forget-Free Winning Subnetworks for Video Representations (TPAMI 2024 · similarity 0.87)
- Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer (NeurIPS 2022 · similarity 0.86)
- A Combinatorial Perspective on Transfer Learning (NeurIPS 2020 · similarity 0.86)
- Task-aware Orthogonal Sparse Network for Exploring Shared Knowledge in Continual Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
