Research map / Continual learning / Online and replay-based
Learning where to learn: Gradient sparsity in meta and continual learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn binary per-parameter gradient masks jointly with the initialization | MAML few-shot learning · miniImageNet 5-way 5-shot, ConvNet | accuracy (%): 63.15$^{\pm0.91}$ -> 67.03$^{\pm0.74}$ | Helped |
| use the learned stochastic-mask meta-optimization variant sparse-MAML+ (ablation) | sparse-MAML · miniImageNet 5-way 5-shot, ConvNet | accuracy (%): 67.03$^{\pm0.74}$ -> 68.05$^{\pm0.84}$ | Helped |
| replace binary masks with rectified learning rates (ablation) | sparse-MAML · miniImageNet 5-way 5-shot, ConvNet | accuracy (%): 67.03$^{\pm0.74}$ -> 64.84$^{\pm0.46}$ | Hurt |
| use sparse-ReLU-MAML instead of binary-mask sparse-MAML (ablation) | few-shot learning with ResNet-12 · miniImageNet 5-way 5-shot, ResNet-12 | accuracy (%): 70.02$^{\pm1.12}$ -> 73.01$^{\pm0.24}$ | Helped |
| meta-learn the freezing pattern rather than manually selecting frozen layers | few-shot learners trained on miniImageNet · cross-domain 5-shot: TieredImageNet, CUB, and Cars | accuracy (%) | Mixed |
| remove bias parameters (ablation) | sparse-MAML · miniImageNet 5-way 5-shot | test accuracy (%): 67.03$^{\pm0.74}$ -> 66.11$^{\pm0.57}$ | Hurt |
| use non-transductive BatchNorm (ablation) | sparse-MAML · miniImageNet 5-way 5-shot | test accuracy (%): 55.58$^{\pm1.68}$ -> 54.75$^{\pm1.17}$ | Hurt |
| learn a gradient mask after MAML training while keeping the initialization fixed (ablation) | MAML few-shot learner · miniImageNet 5-way 5-shot, two-phase training | test accuracy (%): MAML alone does not improve; 64.68$^{\pm0.16}$ and 64.01$^{\pm0.47}$ | Hurt |
| replace La-MAML's learned learning rates with dynamically trainable binary gradient masks (ablation) | La-MAML · MNIST rotations, permutations, and many permutations | retained accuracy (RA): 77.42$^{\pm 0.65}$ -> 77.77$^{\pm 0.58}$; 74.34$^{\pm 0.67}$ -> 76.88$^{\pm 0.72}$; 48.46$^{\pm 0.45}$ -> 50.81$^{\pm 0. | Helped |
| use binary masks with sparse-C-MAML (ablation) | continual-MAML · Omniglot-MNIST-FashionMNIST online learning | cumulative online accuracy: 84.5$^{\pm1.7}$ -> 94.2$^{\pm0.4}$ (p=0.98); 75.5\std{0.7} -> 86.3\std{0.4} (p=0.9) | Helped |
| replace sparse-La-MAML masks with rectified learning rates (ablation) | sparse-C-MAML · Omniglot-MNIST-FashionMNIST online learning | cumulative online accuracy: 94.2$^{\pm0.4}$ -> 93.5\std{0.5} (p=0.98); 86.3\std{0.4} -> 86.1\std{0.2} (p=0.9) | Hurt |
| use sparse-La-MAML instead of La-MAML (ablation) | continual learning · Split-CIFAR-10 with replay memory 200 and 1000 | final accuracy (%): 22.08$^{\pm1.83}$ -> 27.85$^{\pm0.69}$; 36.24$^{\pm0.91}$ -> 37.70$^{\pm0.80}$ | Helped |
Closest papers by meaning
- Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm (ICLR 2018 · similarity 0.89)
- On the Stability and Generalization of Meta-Learning: the Impact of Inner-Levels (NeurIPS 2025 · similarity 0.89)
- On the Stability and Generalization of Meta-Learning (NeurIPS 2024 · similarity 0.88)
- Continuous-Time Meta-Learning with Forward Mode Differentiation (ICLR 2022 · similarity 0.88)
- Memory-Reduced Meta-Learning with Guaranteed Convergence (AAAI 2025 · similarity 0.87)
- Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.