Research map / Continual learning / Online and replay-based
Parameter-Level Soft-Masking for Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add parameter-level soft-masking (SPG) instead of hard neuron masks (HAT) | continual learning on dissimilar tasks · Dissimilar tasks avg | Accuracy avg (%): 61.8 vs 55.4 for HAT | Helped |
| Add parameter-level soft-masking (SPG) instead of hard neuron masks (HAT) | continual learning on similar tasks · Similar tasks avg | Accuracy avg (%): 87.4 vs 82.3 for HAT | Helped |
| Replace EWC-style regularization with soft-masking using gradient importance (ablation) | SPG vs EWC-GI · Dissimilar tasks avg | Accuracy avg (%): 61.8 vs 54.6 | Helped |
| Use gradient-based importance instead of Fisher information for soft-masking (ablation) | SPG vs SPG-FI · Dissimilar tasks avg | Accuracy avg (%): 61.8 vs 54.4 | Helped |
| Add cross-head importance (CHI) (ablation) | SPG · Dissimilar tasks avg | Accuracy avg (%): 61.8 vs 59.1 w/o CHI | Helped |
| Add cross-head importance (CHI) (ablation) | SPG · Similar tasks avg | Accuracy avg (%): 87.4 vs 87.3 w/o CHI | No clear effect |
| Add soft-masking of classification head (SMH) (ablation) | SPG · Dissimilar tasks avg | Accuracy avg (%): 61.8 vs 60.0 w/o SMH | Helped |
| Add soft-masking of classification head (SMH) (ablation) | SPG · Similar tasks avg | Accuracy avg (%): 87.4 vs 86.8 w/o SMH | Helped |
| Replace soft-masking with hard binary masks (threshold) (ablation) | SPG · Dissimilar tasks avg | Accuracy avg (%): 61.8 vs 57.7 hard-masking | Hurt |
| Replace soft-masking with hard binary masks (threshold) (ablation) | SPG · Similar tasks avg | Accuracy avg (%): 87.4 vs 86.3 hard-masking | Hurt |
| Prune highest-importance parameters (gradient importance) in first task (ablation) | first task model · C-10 accuracy | Accuracy (%): Highest 10%: 19.4 vs Nothing 75.5 | Hurt |
Closest papers by meaning
- Does Continual Learning Equally Forget All Parameters? (ICML 2023 · similarity 0.89)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.89)
- Model Sensitivity Aware Continual Learning (NeurIPS 2024 · similarity 0.88)
- When to Learn and When to Stop: Quitting at the Optimal Time (Student Abstract) (AAAI 2025 · similarity 0.88)
- Task-aware Orthogonal Sparse Network for Exploring Shared Knowledge in Continual Learning (ICML 2024 · similarity 0.88)
- Learn more, but bother less: parameter efficient continual learning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.