AI Paper Atlas

Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks

Daiki Chijiwa, Shin'ya Yamaguchi, Atsutoshi Kumagai, Yasutoshi Ida

NeurIPS 2022 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Meta-learning analysis and benchmarks · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Meta-learn a sparse mask at random initialization instead of meta-learning initial weights5-MLP · 5-shot cross-domain classification on CIFAR-FS, VGG-Flower and Aircraftaccuracy: 53.48 ± 0.36 % -> 55.51 ± 0.47 %; 58.60 ± 0.59 % -> 62.34 ± 0.17 %; 28.95 ± 0.76 % -> 31.85 ± 0.31 %Improved
Meta-learn a sparse mask at random initialization instead of meta-learning initial weightsWideResNet-28-10 · 5-shot miniImageNet-to-CUB/Cars and in-domain miniImageNetaccuracy: 62.83 ± 2.47 % -> 74.63 ± 0.22 %; 46.46 ± 3.54 % -> 63.26 ± 0.86 %; 37.09 ± 2.49 % -> 52.54 ± 0.70 %Improved
Set the output-layer inner learning rate to zero (BOIL modification) (ablation)Meta-ticket · 5-shot cross-domain classification with 5-MLP and CNNsaccuracy: 5-MLP: 55.51 -> 56.06, 62.34 -> 63.31, 31.85 -> 33.02; ResNet-12: 71.31 -> 74.23, 57.97 -> 64.06, 45.90 -> 55.20; WideReMixed
Use larger networks rather than small-width networksMeta-ticket · Omniglot 5-shot 20-way classificationaccuracyImproved
Apply iterative randomization to Meta-ticket (ablation)Meta-ticket · Few-shot learning with small neural networksaccuracyImproved
Learn the subnetwork mask with all network parameters fixed to the same constant, rather than relying on parameter signals (ablation)Meta-ticket · Omniglot 5-shot 5-wayaccuracy: about 33% -> 71.59 ± 1.51 %Improved
Use lower initial sparsity (ablation)Meta-ticket with ResNet-12 · miniImageNet 1-shot 5-way meta-validationaccuracy: 54.70 ± 1.90 % (0.0) vs 53.40 ± 0.50 % (0.3), 52.27 ± 0.84 % (0.5), 51.57 ± 0.64 % (0.7), 51.53 ± 0.92 % (0.9)Improved
Increase the score-parameter outer learning rate from 0.01 to 10.0 (ablation)Meta-ticket with ResNet-12 · miniImageNet 1-shot 5-way meta-validationaccuracy: 34.87 ± 1.40% -> 54.30 ± 2.52%Improved
Use Meta-ticket instead of MAMLResNet-12 · miniImageNet 1-shot 5-wayaccuracy: 56.25 ± 1.28 % -> 56.17 ± 0.91 %No clear effect
Use Meta-ticket instead of MAMLWideResNet-28-10 · miniImageNet 1-shot 5-wayaccuracy: 50.59 ± 0.54 % -> 54.12 ± 1.24 %Improved
Use Meta-ticket instead of MAMLMeta-ticket regression configuration · Sinusoid regression 5-shotMSE: 0.346 ± 0.113 -> 3.56 ± 0.11Hurt
Meta-optimize input/output layer weights and use a sparse mask only in the intermediate layer (ablation)Meta-ticket on sinusoid regression · Sinusoid regression 5-shotMSE: 3.56 ± 0.11 -> 0.596 ± 0.173Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy