Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks
Daiki Chijiwa, Shin'ya Yamaguchi, Atsutoshi Kumagai, Yasutoshi Ida
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Meta-learn a sparse mask at random initialization instead of meta-learning initial weights | 5-MLP · 5-shot cross-domain classification on CIFAR-FS, VGG-Flower and Aircraft | accuracy: 53.48 ± 0.36 % -> 55.51 ± 0.47 %; 58.60 ± 0.59 % -> 62.34 ± 0.17 %; 28.95 ± 0.76 % -> 31.85 ± 0.31 % | Improved |
| Meta-learn a sparse mask at random initialization instead of meta-learning initial weights | WideResNet-28-10 · 5-shot miniImageNet-to-CUB/Cars and in-domain miniImageNet | accuracy: 62.83 ± 2.47 % -> 74.63 ± 0.22 %; 46.46 ± 3.54 % -> 63.26 ± 0.86 %; 37.09 ± 2.49 % -> 52.54 ± 0.70 % | Improved |
| Set the output-layer inner learning rate to zero (BOIL modification) (ablation) | Meta-ticket · 5-shot cross-domain classification with 5-MLP and CNNs | accuracy: 5-MLP: 55.51 -> 56.06, 62.34 -> 63.31, 31.85 -> 33.02; ResNet-12: 71.31 -> 74.23, 57.97 -> 64.06, 45.90 -> 55.20; WideRe | Mixed |
| Use larger networks rather than small-width networks | Meta-ticket · Omniglot 5-shot 20-way classification | accuracy | Improved |
| Apply iterative randomization to Meta-ticket (ablation) | Meta-ticket · Few-shot learning with small neural networks | accuracy | Improved |
| Learn the subnetwork mask with all network parameters fixed to the same constant, rather than relying on parameter signals (ablation) | Meta-ticket · Omniglot 5-shot 5-way | accuracy: about 33% -> 71.59 ± 1.51 % | Improved |
| Use lower initial sparsity (ablation) | Meta-ticket with ResNet-12 · miniImageNet 1-shot 5-way meta-validation | accuracy: 54.70 ± 1.90 % (0.0) vs 53.40 ± 0.50 % (0.3), 52.27 ± 0.84 % (0.5), 51.57 ± 0.64 % (0.7), 51.53 ± 0.92 % (0.9) | Improved |
| Increase the score-parameter outer learning rate from 0.01 to 10.0 (ablation) | Meta-ticket with ResNet-12 · miniImageNet 1-shot 5-way meta-validation | accuracy: 34.87 ± 1.40% -> 54.30 ± 2.52% | Improved |
| Use Meta-ticket instead of MAML | ResNet-12 · miniImageNet 1-shot 5-way | accuracy: 56.25 ± 1.28 % -> 56.17 ± 0.91 % | No clear effect |
| Use Meta-ticket instead of MAML | WideResNet-28-10 · miniImageNet 1-shot 5-way | accuracy: 50.59 ± 0.54 % -> 54.12 ± 1.24 % | Improved |
| Use Meta-ticket instead of MAML | Meta-ticket regression configuration · Sinusoid regression 5-shot | MSE: 0.346 ± 0.113 -> 3.56 ± 0.11 | Hurt |
| Meta-optimize input/output layer weights and use a sparse mask only in the intermediate layer (ablation) | Meta-ticket on sinusoid regression · Sinusoid regression 5-shot | MSE: 3.56 ± 0.11 -> 0.596 ± 0.173 | Improved |
Closest papers by meaning
- Meta-Learning of Neural Architectures for Few-Shot Learning (CVPR 2020 · similarity 0.89)
- Meta-Learning with Network Pruning (ECCV 2020 · similarity 0.88)
- Meta-Transfer Learning Through Hard Tasks (TPAMI 2020 · similarity 0.88)
- Meta-Transfer Learning for Few-Shot Learning (CVPR 2019 · similarity 0.88)
- Meta-Learning With Differentiable Convex Optimization (CVPR 2019 · similarity 0.88)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
