Research map / Architectures and efficiency / RNNs and normalization
Improving Accuracy of Binary Neural Networks Using Unbalanced Activation Distribution
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a 0.6 binary-activation threshold shift | BNN (AlexNet) · ImageNet | Top-1 accuracy: 41.5 -> 42.1 | Helped |
| Add a 0.8 binary-activation threshold shift | XNOR-Net (AlexNet) · ImageNet | Top-1 accuracy: 44.4 -> 45.6 | Helped |
| Add a 2.0 binary-activation threshold shift | XNOR-Net (ResNet-18) · ImageNet | Top-1 accuracy: 51.2 -> 54.2 | Helped |
| Add threshold shifts of 2.0 for the pretrained model and 1.6 for the BNN | Bi-Real-Net (ResNet-18) · ImageNet | Top-1 accuracy: 56.1 -> 57.2 | Helped |
| Add threshold shifts of 2.0 for the pretrained model and 1.8 for the BNN | Bi-Real-Net (ResNet-34) · ImageNet | Top-1 accuracy: 61.9 -> 62.8 | Helped |
| Shift hardtanh along the x-axis to make its output distribution skewed (ablation) | vgg-small · CIFAR-10 test | accuracy | Helped |
| Shift hardtanh along the y-axis or increase its range instead of shifting along the x-axis (ablation) | vgg-small · CIFAR-10 test | accuracy | No clear effect |
| Replace Max Pooling with Average Pooling (ablation) | binary vgg-small · CIFAR-10 test | accuracy | Mixed |
| Use a positive rather than negative threshold shift (ablation) | binary vgg-small with Max Pooling · CIFAR-10 test | accuracy | Helped |
| Use separately tuned shifts for the pretrained model and BNN (ablation) | vgg-small initialized from a full-precision pretrained model · MNIST | accuracy | Helped |
| Train the binary activation thresholds using back-propagation rather than fixing the shift (ablation) | vgg-small BNN · CIFAR-10 test | accuracy | No clear effect |
| Add PReLU layers after binary convolution blocks (ablation) | ResNet-20 with double skip connections · CIFAR-10 test | accuracy | Helped |
Closest papers by meaning
- Binarized Neural Networks (NeurIPS 2016 · similarity 0.88)
- BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations (ICLR 2020 · similarity 0.88)
- Regularizing Activation Distribution for Training Binarized Deep Networks (CVPR 2019 · similarity 0.88)
- How to Train a Compact Binary Neural Network with High Accuracy? (AAAI 2017 · similarity 0.87)
- Towards Accurate Binary Convolutional Neural Network (NeurIPS 2017 · similarity 0.87)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.