Research map / Architectures and efficiency / RNNs and normalization
Regularizing Activation Distribution for Training Binarized Deep Networks
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add activation distribution loss | BNN · CIFAR-10 | accuracy: 87.13% -> 89.90% | Helped |
| Add activation distribution loss | BNN · SVHN | accuracy: 96.50% -> 97.23% | Helped |
| Add activation distribution loss | BNN · CIFAR-100 | accuracy: 60.40% -> 68.17% | Helped |
| Add activation distribution loss | BNN · ImageNet ILSVRC-2012 (AlexNet), BNN | Top-1 accuracy: 36.1% -> 41.3% | Helped |
| Add activation distribution loss | BNN · ImageNet ILSVRC-2012 (AlexNet), WRPN | Top-1 accuracy: 48.3% -> 53.8% | Helped |
| Use BNN-DL instead of BNN with Momentum | CIFAR-10 · CIFAR-10, Momentum optimizer | accuracy: 66.02% -> 89.37% | Helped |
| Use BNN-DL instead of BNN with Nesterov | CIFAR-10 · CIFAR-10, Nesterov optimizer | accuracy: 68.66% -> 89.22% | Helped |
| Use BNN-DL instead of BNN with Adam | CIFAR-10 · CIFAR-10, Adam optimizer | accuracy: 88.12% -> 89.62% | Helped |
| Use BNN-DL instead of BNN with RMSprop | CIFAR-10 · CIFAR-10, RMSprop optimizer | accuracy: 87.39% -> 90.24% | Helped |
| Use BNN-DL instead of BNN at initial learning rate 1e-1 | CIFAR-10 · CIFAR-10, lr_init=1e-1 | accuracy: 82.19% -> 89.60% | Helped |
| Vary the distribution-loss coefficient λ from 0 to 0.2–2000 (ablation) | BNN-DL · CIFAR-10 | accuracy: 87.39 -> 90.12; 90.16; 90.18; 90.61; 90.20 | Helped |
| Use BNN-DL instead of BNN with ResNet-18 | CIFAR-10 · CIFAR-10, ResNet-18 | accuracy: 85.71% -> 90.47% | Helped |
Closest papers by meaning
- Loss-aware Binarization of Deep Networks (ICLR 2017 · similarity 0.88)
- Improving Accuracy of Binary Neural Networks Using Unbalanced Activation Distribution (CVPR 2021 · similarity 0.88)
- Binarized Neural Networks (NeurIPS 2016 · similarity 0.88)
- How to Train a Compact Binary Neural Network with High Accuracy? (AAAI 2017 · similarity 0.86)
- Forward and Backward Information Retention for Accurate Binary Neural Networks (CVPR 2020 · similarity 0.86)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.