Research map / Architectures and efficiency / Neural architecture search
PreNAS: Preferred One-Shot Learning Towards Efficient Neural Architecture Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use preferred one-shot training on the preselected architectures instead of redundant training on the full space (ablation) | PreNAS · ImageNet, Tiny search space | Top-1 accuracy (%): about 0.5% | Helped |
| Use preferred one-shot training on the preselected architectures instead of redundant training on the full space (ablation) | PreNAS · ImageNet, Small search space | Top-1 accuracy (%): about 0.5% | Helped |
| Add layer normalization to SNIP when ranking architectures within Transformer isomer groups (ablation) | PreNAS zero-cost selector · Case 1, Transformer isomers | mBR: 0.986 -> 0.099 | Helped |
| Add layer normalization to SNIP when ranking architectures within Transformer isomer groups (ablation) | PreNAS zero-cost selector · Case 2, Transformer isomers | mBR: 0.825 -> 0.180 | Helped |
| Replace balanced sampling with uniform sampling over the preferred space (ablation) | PreNAS one-shot training · ImageNet, Tiny search space | subnet accuracy | Hurt |
| Replace balanced sampling with resampling from decomposed variable factors (ablation) | PreNAS one-shot training · ImageNet, Tiny search space | subnet accuracy | Mixed |
| Use balanced sampling over embedding dimension and depth (ablation) | PreNAS one-shot training · ImageNet, Tiny search space | subnet accuracy | Helped |
| Use the normalized isomer proxy instead of original SNIP for selecting representatives (ablation) | PreNAS zero-cost selector · Transformer search space, refined search spaces | error rate distribution | Helped |
| Apply PreNAS preferred learning to the BigNAS CNN search space | BigNAS CNN · ImageNet, CNN models | Top-1 accuracy (%): from 0.5% to 1.0% | Helped |
| Use PreNAS-Ti instead of LVT | Vision Transformer · ImageNet | Top-1 accuracy (%): 74.8 -> 77.1 | Helped |
| Use PreNAS-S instead of BoTNet-S1-59 | Vision Transformer · ImageNet | Top-1 accuracy (%): 81.7 -> 81.8 | Helped |
| Use PreNAS-B instead of BoTNet-S1-110 | Vision Transformer · ImageNet | Top-1 accuracy (%): 82.8 -> 82.6 | Hurt |
Closest papers by meaning
- Few-Shot Neural Architecture Search (ICML 2021 · similarity 0.91)
- Overcoming Multi-Model Forgetting in One-Shot NAS With Diversity Maximization (CVPR 2020 · similarity 0.91)
- Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (AAAI 2025 · similarity 0.89)
- Neural Fine-Tuning Search for Few-Shot Learning (ICLR 2024 · similarity 0.88)
- Revisiting Neural Networks for Few-Shot Learning: A Zero-Cost NAS Perspective (ICML 2025 · similarity 0.88)
- L-SWAG: Layer-Sample Wise Activation with Gradients Information for Zero-Shot NAS on Vision Transformers (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.