Research map / Label-efficient and robust learning / OOD and anomaly detection
Two Sides of Meta-Learning Evaluation: In vs. Out of Distribution
Amrith Setlur, Oscar Li, Virginia Smith
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train PN with more ways than test (64-way vs 5-way) (ablation) | ProtoNet · miniImageNet-mod | NovelGen vs BaseGen accuracy | Mixed |
| Use higher learning rate for FOMAML (ablation) | FOMAML · FC100-mod | BaseGen accuracy | Improved |
| Use lower learning rate for FOMAML (ablation) | FOMAML · FC100-mod | NovelGen accuracy | Improved |
| Fix support examples per base class (FixML) (ablation) | SVM, Ridge, PN · CIFAR-FS-mod | NovelGen accuracy | Improved |
| Fix support examples per base class (FixML) (ablation) | SVM, Ridge, PN · CIFAR-FS-mod | BaseGen accuracy | Hurt |
| Reduce Zappos-ID training tasks from 1000 to 50 (ablation) | PN vs SVM · Zappos-ID 2w5s | accuracy gap PN over SVM: 86.58 -> 77.67 | Improved |
| Select snapshot by best ValGen instead of best NovelGen | SVM, Ridge, PN, FOMAML · FC100 | NovelGen accuracy: 1.1, 1.2, 0.7, 0.9 lost | Hurt |
| Select snapshot by BaseGen instead of ValGen | SVM · CIFAR-FS-mod | NovelGen accuracy | Hurt |
| Select snapshot by BaseGen instead of ValGen | PN · tieredImageNet-mod | NovelGen accuracy | Improved |
| Use ID validation (Zappos-ID, supervised IC) instead of OOD validation | snapshot selection · FSL benchmarks | Kendall rho | Improved |
| Use 160 novel classes instead of 20 for evaluation | method comparison · miniImageNet/tieredImageNet | conclusion flip and exaggeration frequency | Improved |
| Train on 351 base classes instead of 64 (tieredImageNet) | PN vs Ridge · tieredImageNet-mod | conclusion flip rate | Improved |
Closest papers by meaning
- OOD-MAML: Meta-Learning for Few-Shot Out-of-Distribution Detection and Classification (NeurIPS 2020 · similarity 0.89)
- Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples (ICLR 2020 · similarity 0.87)
- Secure Out-of-Distribution Task Generalization with Energy-Based Models (NeurIPS 2023 · similarity 0.87)
- MetaOOD: Automatic Selection of OOD Detection Models (ICLR 2025 · similarity 0.87)
- OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable? (ICML 2025 · similarity 0.85)
- Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
