Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
MAML and ANIL Provably Learn Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt the task-specific head before the outer-loop representation update | MAML and ANIL · Multi-task linear representation learning, infinite samples | principal angle distance to the ground-truth subspace | Helped |
| Use task-specific head adaptation instead of optimizing the average non-adaptive loss (ablation) | Multi-task linear representation learning · Task population losses | principal angle distance to the ground-truth subspace | Helped |
| Freeze the representation during inner-loop adaptation (ablation) | MAML (ANIL variant) · Multi-task linear representation learning, infinite samples | principal angle distance to the ground-truth subspace | Helped |
| Adapt the representation in the inner loop (ablation) | MAML compared with ANIL · Multi-task linear representation learning, infinite samples | principal angle distance to the ground-truth subspace | Mixed |
| Replace exact second-order updates with first-order updates (ablation) | ANIL · Multi-task linear representation learning, infinite samples | principal angle distance to the ground-truth subspace | Hurt |
| Replace exact second-order updates with first-order updates (ablation) | MAML · Multi-task linear representation learning, infinite samples | principal angle distance to the ground-truth subspace | Mixed |
| Increase task diversity | ANIL and MAML · Multi-task linear representation learning | rate of principal angle contraction | Helped |
| Use a less accurate initialization (ablation) | MAML · Multi-task linear representation learning, low task diversity | principal angle convergence | Hurt |
| Use a nonzero mean for the ground-truth task heads (ablation) | FO-MAML · Multi-task linear representation learning, low task diversity | principal angle convergence | Hurt |
| Center the ground-truth task heads around zero (ablation) | FO-MAML · Multi-task linear representation learning | principal angle convergence | Helped |
| Use finite rather than infinite samples per task (ablation) | ANIL and FO-ANIL · Multi-task linear representation learning, finite samples | principal angle distance to the ground-truth subspace | Hurt |
| Use exact rather than first-order ANIL updates (ablation) | ANIL · Multi-task linear representation learning, finite samples | samples required per task | Hurt |
Closest papers by meaning
- Statistically and Computationally Efficient Linear Meta-representation Learning (NeurIPS 2021 · similarity 0.90)
- Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML (ICLR 2020 · similarity 0.90)
- BOIL: Towards Representation Change for Few-shot Learning (ICLR 2021 · similarity 0.90)
- First-order ANIL provably learns representations despite overparametrisation (ICLR 2024 · similarity 0.90)
- Provable Meta-Learning of Linear Representations (ICML 2021 · similarity 0.89)
- How Fine-Tuning Allows for Effective Meta-Learning (NeurIPS 2021 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.