Research map / Continual learning / Online and replay-based
Distillation-Guided Structural Transfer for Continual Learning Beyond Sparse Distributed Memory
Huiyan Xue, Xuming Ran, Yaxin Li, Qi Xu, Enhui Li, Yi Xu, Qiang Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Selective Subnetwork Distillation | SDMLP · Split CIFAR-10 (10k neurons, K=10) | validation accuracy: 0.71 -> 0.81 | Improved |
| add Selective Subnetwork Distillation | SDMLP · Split CIFAR-10 (1k neurons, K=10) | validation accuracy: 0.63 -> 0.73 | Improved |
| add Selective Subnetwork Distillation | SDMLP · Split CIFAR-100 (10k neurons, K=10) | validation accuracy: 0.32 -> 0.43 | Improved |
| add Selective Subnetwork Distillation | SDMLP · Split MNIST (10k neurons, K=10) | validation accuracy: 0.53 -> 0.57 | Improved |
| combine SSD with EWC | SSD · Split CIFAR-10 (10k neurons) | validation accuracy: 0.81 -> 0.87 | Improved |
| combine SSD with EWC | SSD · Split MNIST (10k neurons, K=10) | validation accuracy: 0.57 -> 0.86 | Improved |
| add SSD to reduce forgetting | SDMLP · Split CIFAR-10 (10k neurons) | mean BWT: -0.1828 -> -0.1234 | Improved |
| add hierarchical (hidden + output) distillation on top of selective (ablation) | SSD · Split CIFAR-10 (10k neurons) | validation accuracy: 0.7874 -> 0.8083 | Improved |
| add selective Top-n filtering on top of hierarchical (ablation) | SSD · Split CIFAR-10 (10k neurons) | validation accuracy: 0.8056 -> 0.8083 | Improved |
| shift KD weight toward output logits (lambda 0.9 -> 0.1) (ablation) | SSD · Split CIFAR-10 (10k neurons) | validation accuracy: 0.7902 -> 0.8174 | Improved |
| vary Top-n between 200 and 1000 (ablation) | SSD · Split CIFAR-10 (10k neurons) | validation accuracy: 0.8081 (n=200) vs 0.8083 (n=1000) | No clear effect |
| use higher distillation temperature (T = 6 or 8) (ablation) | SSD · Split CIFAR-10 (10k neurons) | validation accuracy: best 0.8174 at alpha 0.7, T 8.0 | Improved |
Closest papers by meaning
- Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning (AAAI 2024 · similarity 0.88)
- InTAct: Interval-based Task Activation Consolidation for Continual Learning (NeurIPS 2026 · similarity 0.88)
- Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective (ICML 2025 · similarity 0.88)
- Task-aware Orthogonal Sparse Network for Exploring Shared Knowledge in Continual Learning (ICML 2024 · similarity 0.88)
- Learning without Isolation: Pathway Protection for Continual Learning (ICML 2025 · similarity 0.87)
- Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
