Research map / Architectures and efficiency / RNNs and normalization
Training Binary Weight Networks via Semi-Binary Decomposition
Qinghao Hu, Gang Li, Peisong Wang, Yifan Zhang, Jian Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| minimize feature-map quantization loss (SBD-FQ) instead of direct weight decomposition (ablation) | semi-binary decomposition · ImageNet-2012 val, AlexNet | top-1 accuracy (%): 58.0 -> 58.5 | Improved |
| minimize feature-map quantization loss (SBD-FQ) instead of direct weight decomposition (ablation) | semi-binary decomposition · ImageNet-2012 val, ResNet-18 | top-1 accuracy (%): 64.9 -> 66.2 | Improved |
| minimize feature-map quantization loss (SBD-FQ) instead of direct weight decomposition (ablation) | semi-binary decomposition · ImageNet-2012 val, ResNet-50 | top-1 accuracy (%): 67.7 -> 68.9 | Improved |
| use semi-binary decomposition (SBD-FQ) instead of BWN scaling-factor binarization | binary weight ResNet-18 · ImageNet-2012 val, ResNet-18 | top-1 accuracy (%): 60.8 -> 66.2 | Improved |
| use semi-binary decomposition (SBD-FQ) instead of BWN | binary weight ResNet-50 · ImageNet-2012 val, ResNet-50 | top-1 accuracy (%): 63.9 -> 68.9 | Improved |
| use semi-binary decomposition (SBD-FQ) instead of BWN | binary weight AlexNet · ImageNet-2012 val, AlexNet | top-1 accuracy (%): 56.8 -> 58.5 | Improved |
| replace BinaryConnect with SBD-FQ | binary weight AlexNet · ImageNet-2012 val, AlexNet | top-1 accuracy (%): 35.4 -> 58.5 | Improved |
| quantize left/right singular vectors of SVD (Binary-SVD) instead of semi-binary alternating optimization (ablation) | semi-binary components for AlexNet · ImageNet-2012 val, AlexNet, no fine-tuning | top-5 accuracy (%) | Hurt |
| learn components with SBD-FQ instead of SBD-Direct when many layers are quantized (ablation) | AlexNet without fine-tuning · ImageNet-2012 val, AlexNet, no fine-tuning | top-5 accuracy (%) | Improved |
| increase beta (smaller K) in SBD-Direct (ablation) | AlexNet without fine-tuning · ImageNet-2012 val, AlexNet, no fine-tuning | top-5 accuracy (%) | Hurt |
| increase beta for fully-connected layers (ablation) | AlexNet fc layers · ImageNet-2012 val, AlexNet, no fine-tuning | top-5 accuracy (%) | No clear effect |
| deploy binary-weight AlexNet with 8-bit activations on FPGA instead of 32-bit floating point | FPGA accelerator · FPGA Xilinx Virtex-7 VX485T | latency (ms): 21.6 -> 2.46 | Improved |
Closest papers by meaning
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.86)
- How to Train a Compact Binary Neural Network with High Accuracy? (AAAI 2017 · similarity 0.85)
- Binarized Neural Networks (NeurIPS 2016 · similarity 0.85)
- Learning Architectures for Binary Networks (ECCV 2020 · similarity 0.85)
- Forward and Backward Information Retention for Accurate Binary Neural Networks (CVPR 2020 · similarity 0.84)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
