Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
FREE: Faster and Better Data-Free Meta-Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FREE, a data-free meta-learning framework with a task-level meta-generator (FIVE) that recovers training tasks from pre-trained models in 5 adaptation steps, and a gradient-alignment meta-learner (BELL) trained via multi-task knowledge distillation.
Problem
Data-free meta-learning must recover training data from a pool of pre-trained models without the original data, but existing inversion is slow (hundreds of generate-forward-backward iterations per model) and ignores the distribution gap among heterogeneous pre-trained models, causing task conflicts.
Approach
Treat each pre-trained model as a task and meta-learn a shared generator that adapts to any model in about five steps (FIVE); then train the meta-learner with an implicit gradient alignment objective across tasks from different models (BELL), plus cross-task replay from a memory bank.
Key results
- FREE5 reaches 39.13% 5-way 1-shot and 52.58% 5-shot on CIFAR-FS, versus 37.66%/51.16% for BiDf-MKD, with data recovery of 0.11h versus 2.47h.
- The method reports about 20x speed-up in data recovery and 1.42%-4.78% 5-shot accuracy gains over the strongest baseline.
- In the multi-domain setting FREE5 reaches 31.51% 1-shot versus 31.44% for BiDf-MKD and 44.04% 5-shot versus 40.96%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FREE5 instead of BiDf-MKD | DFML with Conv4 · CIFAR-FS 5-way 1-shot | accuracy: 37.66 ± 0.75 -> 39.13 ± 0.85 | Helped |
| Use FREE5 instead of BiDf-MKD | DFML with Conv4 · CIFAR-FS 5-way 5-shot | accuracy: 51.16 ± 0.79 -> 52.58 ± 0.77 | Helped |
| Use FREE5 instead of BiDf-MKD | DFML with Conv4 · miniImageNet 5-way 1-shot | accuracy: 30.66 ± 0.59 -> 33.03 ± 0.69 | Helped |
| Use FREE5 instead of BiDf-MKD | DFML with Conv4 · miniImageNet 5-way 5-shot | accuracy: 42.30 ± 0.64 -> 45.45 ± 0.69 | Helped |
| Use FREE5 instead of BiDf-MKD | DFML with Conv4 · CUB 5-way 1-shot | accuracy: 31.62 ± 0.60 -> 31.94 ± 0.61 | Helped |
| Use FREE5 instead of BiDf-MKD | DFML with Conv4 · CUB 5-way 5-shot | accuracy: 44.32 ± 0.69 -> 49.10 ± 0.68 | Helped |
| Add FIVE and BELL to the cross-task-replay baseline (ablation) | FREE5 · CIFAR-FS 5-way 1-shot | accuracy: 36.33 ± 0.81 -> 39.13 ± 0.85 | Helped |
| Add FIVE and BELL to the cross-task-replay baseline (ablation) | FREE5 · CIFAR-FS 5-way 5-shot | accuracy: 47.67 ± 0.75 -> 52.58 ± 0.77 | Helped |
Surprising findings
- Plain ERM over recovered tasks performs worse than the approach as the number of pre-trained models grows, and can even beat it with a small model pool.
Limitations
- Time cost of meta-learner training is omitted from speed comparisons.
- Evaluated only with Conv4 and small ResNets on three few-shot image benchmarks at small scale.
Experimental setup
Datasets: miniImageNet CIFAR-FS CUB
Benchmarks: miniImageNet CIFAR-FS CUB
Metrics: 5-way 1-shot accuracy 5-way 5-shot accuracy GPU hours for data recovery
Baselines: Random Best-Model Average OTA DRO PURER BiDf-MKD
Builds on: DeepInversion MAML Reptile PURER BiDf-MKD
Closest papers by meaning
- Efficient Source-free Unlearning via Energy-Guided Data Synthesis and Discrimination-Aware Multitask Optimization (ICML 2025 · similarity 0.81)
- Memory-Reduced Meta-Learning with Guaranteed Convergence (AAAI 2025 · similarity 0.80)
- MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters (ICML 2025 · similarity 0.80)
- DataRater: Meta-Learned Dataset Curation (NeurIPS 2025 · similarity 0.80)
- Toward Efficient Data-Free Unlearning (AAAI 2025 · similarity 0.80)
- Towards Source-Free Machine Unlearning (CVPR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.