Research map / Architectures and efficiency / Neural architecture search
ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| heterogeneous weight sharing (HWS) instead of naive weight sharing (ablation) | ShiftAddNAS supernet · ImageNet-1k | Top-1 Acc.: +1.4% (82.7% vs 81.3%) | Helped |
| heterogeneous weight sharing (HWS) instead of naive weight sharing (ablation) | ShiftAddNAS supernet · ImageNet-1k | Kendall tau ranking correlation: 0.54 vs 0.49 | Helped |
| heterogeneous weight sharing (HWS) reduces supernet size (ablation) | ShiftAddNAS supernet · ImageNet-1k | supernet size (params): 615M to 364M (41% savings) | Helped |
| hybrid search space (Attn, Conv, Shift, Add) instead of Attn&Conv only (BossNAS-style) (ablation) | ShiftAddNAS search space · ImageNet-1k | Top-1 Acc.: 82.6% vs 81.6% | Helped |
| Shift&Add-only search space instead of full hybrid space (ablation) | ShiftAddNAS search space · ImageNet-1k | Top-1 Acc.: 76.8% vs 82.6% | Hurt |
| adding Shift and Add operators to the FBNet search space (hybrid space) | FBNet search space · CIFAR-100 | Accuracy: +0.58% (78.64% vs 77.86%) | Helped |
| adding Shift and Add operators to the FBNet search space (hybrid space) | FBNet search space · CIFAR-10 | Accuracy: +0.74% (95.83% vs 95.09%) | Helped |
| ShiftAddNAS hybrid search with 8-bit quantization vs full precision on WMT En-Fr | ShiftAddNAS (8-bit) · WMT'14 En-Fr | BLEU: 41.5 vs 39.6 for Lite Transformer (8-bit) | Helped |
| ShiftAddNAS search vs HAT baseline with full precision on WMT En-De | ShiftAddNAS · WMT'14 En-De | BLEU: 28.2 vs 28.2 at 43M params vs 44M | No clear effect |
Closest papers by meaning
- ShiftAddNet: A Hardware-Inspired Deep Network (NeurIPS 2020 · similarity 0.92)
- An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks (ICML 2025 · similarity 0.85)
- Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (AAAI 2025 · similarity 0.85)
- FlashRNN: I/O-Aware Optimization of Traditional RNNs on modern hardware (ICLR 2025 · similarity 0.84)
- ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization (NeurIPS 2024 · similarity 0.84)
- S$^2$NN: Sub-bit Spiking Neural Networks (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.