Research map / Architectures and efficiency / RNNs and normalization
Binarized Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with binary weights and activations rather than full-precision weights and activations | BNN · MNIST, SVHN, and CIFAR-10 | test error rate | Mixed |
| Use the SWAR-based XNOR kernel instead of the unoptimized baseline kernel | MNIST BNN · MNIST test set inference | speedup: 7 times faster | Helped |
| Use the SWAR-based XNOR kernel instead of the unoptimized baseline kernel | Binary matrix multiplication · 8192 × 8192 × 8192 matrix multiplication | speedup: 23 times faster | Helped |
| Use the SWAR-based XNOR kernel instead of cuBLAS | Binary matrix multiplication · 8192 × 8192 × 8192 matrix multiplication | speedup: 3.4 times faster | Helped |
| Replace vanilla batch normalization with shift-based batch normalization (ablation) | BNN · Experiments reported in the paper | accuracy | No clear effect |
| Replace vanilla ADAM with shift-based AdaMax (ablation) | BNN · Experiments reported in the paper | accuracy | No clear effect |
| Keep the straight-through-estimator gradient active when the pre-quantized value is outside [-1, 1] (ablation) | BNN training · Experiments reported in the paper | performance | Hurt |
| Use binary activations and weights in the XNOR kernel rather than non-binary inputs (ablation) | XNOR GPU kernel · GPU matrix multiplication | output equivalence | No clear effect |
Closest papers by meaning
- Sparsity-Inducing Binarized Neural Networks (AAAI 2020 · similarity 0.89)
- Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization (NeurIPS 2019 · similarity 0.89)
- Binary Graph Neural Networks (CVPR 2021 · similarity 0.89)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.88)
- How to Train a Compact Binary Neural Network with High Accuracy? (AAAI 2017 · similarity 0.88)
- A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.