Research map / Architectures and efficiency / CNN design
AdderNet: Do We Really Need Multiplications in Deep Learning?
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace convolution operations with ℓ1-distance adder filters | VGG-small · CIFAR-10 and CIFAR-100 | accuracy: 93.80% -> 93.72%; 72.73% -> 72.64% | No clear effect |
| Replace convolution operations with ℓ1-distance adder filters | ResNet-20 · CIFAR-10 and CIFAR-100 | accuracy: 92.25% -> 91.84%; 68.14% -> 67.60% | Hurt |
| Replace convolution operations with ℓ1-distance adder filters | ResNet-32 · CIFAR-10 and CIFAR-100 | accuracy: 93.29% -> 93.01%; 69.74% -> 69.02% | Hurt |
| Replace convolution operations with adder filters | ResNet-18 · ImageNet | Top-1 accuracy: 69.8% -> 67.0% | Hurt |
| Replace convolution operations with adder filters | ResNet-50 · ImageNet | Top-1 accuracy: 76.2% -> 74.9% | Hurt |
| Use full-precision gradient instead of sign gradient (ablation) | AdderNet with unchanged learning rate · MNIST | accuracy: 29.26% -> 54.91% | Helped |
| Use adaptive learning rate scaling instead of a fixed 100x learning-rate increase (ablation) | AdderNet with full-precision gradient · MNIST | accuracy: 98.99% -> 99.40% | Helped |
| Use sign gradient instead of full-precision gradient (ablation) | AdderNet with adaptive learning rate · MNIST | accuracy: 99.40% -> 97.99% | Hurt |
| Vary η in adaptive learning-rate scaling (ablation) | AdderNet / LeNet-5-BN · MNIST | accuracy: η: 1 -> 99.26; 0.5 -> 99.30; 0.2 -> 99.35; 0.1 -> 99.40; 0.05 -> 99.32 | Mixed |
| Use ℓ2 distance instead of ℓ1 distance (ablation) | AdderNet / ResNet-18 · ImageNet | Top-1 accuracy: 66.8% -> 69.6% | Helped |
Closest papers by meaning
- Towards Stable and Robust AdderNets (NeurIPS 2021 · similarity 0.89)
- Redistribution of Weights and Activations for AdderNet Quantization (NeurIPS 2022 · similarity 0.88)
- Winograd Algorithm for AdderNet (ICML 2021 · similarity 0.88)
- Adder Attention for Vision Transformer (NeurIPS 2021 · similarity 0.86)
- StrassenNets: Deep Learning with a Multiplication Budget (ICML 2018 · similarity 0.86)
- Convolutional Rectifier Networks as Generalized Tensor Decompositions (ICML 2016 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.