Research map / Probabilistic and scientific modeling / Variational and generative sampling
Meta-Learning Universal Priors Using Non-Injective Change of Variables
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MetaNCoV, a meta-learning method that learns a data-driven prior via a non-injective change-of-variable (Sylvester NCoV) flow in latent space, with a universal approximation theorem and gains on few-shot image classification.
Problem
Meta-learning priors are restricted to preselected pdfs (e.g., Gaussian), limiting expressiveness and hurting performance on tasks with extremely scarce data.
Approach
Replace injective normalizing flows with a non-injective Sylvester change-of-variable model, generalizing the probability integral transform to arbitrary target pdfs, and learn the prior by optimizing latent vectors with MAP inner-loop updates, so the zero-latent maps to an implicit task-invariant initialization.
Key results
- MetaSGD + MetaNCoV reaches 59.10% 1-shot and 71.48% 5-shot on 5-way miniImageNet with a 4-block CNN, above the best listed baseline MC at 54.08% 1-shot.
- MC + MetaNCoV reaches 63.40% 1-shot on miniImageNet and 72.38% 1-shot on tieredImageNet with WRN-28-10 features, improving over MC (61.22% and 66.20%).
- On cross-domain tests (miniImageNet to CUB, 5-way 1-shot), MetaNCoV reaches 47.84% versus 44.74% for GAP, the strongest baseline listed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add an NCoV prior to MetaSGD | MetaSGD · 5-class miniImageNet 1-shot | accuracy: 50.47±1.87 -> 59.10±1.52 | Helped |
| add an NCoV prior to MetaSGD | MetaSGD · 5-class miniImageNet 5-shot | accuracy: 64.03±0.94 -> 71.48±0.68 | Helped |
| add an NCoV prior to MC | MC · 5-class tieredImageNet 1-shot, center crop | accuracy: 66.20±0.10 -> 72.38±1.26 | Helped |
| add an NCoV prior to MC | MC · 5-class tieredImageNet 5-shot, center crop | accuracy: 82.21±0.08 -> 86.47±0.56 | Helped |
| add an NCoV prior to MetaSGD | MetaSGD · 5-class CUB-200-2011 1-shot | accuracy: 58.13±0.36 -> 69.94±1.34 | Helped |
| use MetaNCoV instead of the strongest listed baseline | cross-domain few-shot learning · 5-class TieredImageNet 1-shot, CUB 1-shot, Cars 1-shot, and their 5-shot settings | accuracy | Helped |
| replace the non-injective NCoV with an injective NF (ablation) | MetaSGD + MetaNCoV · 5-class miniImageNet 1-shot | accuracy: 59.10±1.52 -> 56.72±1.46 | Hurt |
| replace the non-injective NCoV with an injective NF (ablation) | MetaSGD + MetaNCoV · 5-class miniImageNet 5-shot | accuracy: 71.48±0.68 -> 69.41±0.68 | Hurt |
| replace sigmoid σ with ReLU (ablation) | MetaSGD + MetaNCoV · 5-class miniImageNet 1-shot | accuracy: 59.10±1.52 -> 56.54±1.46 | Hurt |
| replace sigmoid σ with ReLU (ablation) | MetaSGD + MetaNCoV · 5-class miniImageNet 5-shot | accuracy: 71.48±0.68 -> 69.84±0.68 | Hurt |
Surprising findings
- Replacing sigmoid with ReLU in the Sylvester NCoV degrades accuracy (59.10% to 56.54% 1-shot), which the authors link to their sigmoidal-activation theory.
Limitations
- NCoV has no closed-form pdf, so the prior density must be estimated numerically (costly for large d) and cannot be trained by maximum likelihood.
- Universal approximation is shown only for one-layer Sylvester NCoVs; multi-layer and other transformations are deferred to remarks.
- Experiments are limited to small CNN and WRN-28-10 backbones on few-shot image classification datasets.
Experimental setup
Datasets: miniImageNet tieredImageNet CUB-200-2011 Cars
Benchmarks: miniImageNet tieredImageNet CUB-200-2011 Cars
Metrics: 5-way 1-shot accuracy (%) 5-way 5-shot accuracy (%) 95% confidence interval
Baselines: MAML MetaSGD Meta-LSTM R2D2 MC Warp-MAML MAML + L2F MeTAL Minimax-MAML LEO MatchingNet ProtoNet
Builds on: MAML MetaSGD Sylvester normalizing flow MC (Meta-learning with Curvature) probability integral transform
Closest papers by meaning
- Meta-Learning Priors Using Unrolled Proximal Networks (ICLR 2024 · similarity 0.88)
- Meta-Learning without Memorization (ICLR 2020 · similarity 0.85)
- MARS: Meta-learning as Score Matching in the Function Space (ICLR 2023 · similarity 0.85)
- Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning (ICLR 2021 · similarity 0.84)
- Provable Meta-Learning with Low-Rank Adaptations (NeurIPS 2025 · similarity 0.83)
- More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.