Research map / Architectures and efficiency / RNNs and normalization
Compacting Binary Neural Networks by Sparse Kernel Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace standard BNN weights with selected kernel-wise sub-codewords using Sparks | ResNet-18 · ImageNet (ILSVRC2012) | Top-1 accuracy (%): 65.9 -> 64.3 | Mixed |
| Use 0.56-bit Sparks instead of 0.55-bit SLBF | ResNet-18 · ImageNet (ILSVRC2012) | Top-1 accuracy (%): 57.7 -> 64.3 | Helped |
| Use 0.56-bit Sparks instead of 0.60-bit FleXOR | ResNet-18 · ImageNet (ILSVRC2012) | Top-1 accuracy (%): 59.8 -> 64.3 | Helped |
| Use kernel-wise rather than channel-wise codewords (ablation) | Sparks selection-based codebook learning · ImageNet, ResNet-18 | Top-1 accuracy (%) | Helped |
| Select codewords jointly from a fixed codebook rather than learn them independently with product quantization (ablation) | Kernel-wise codebook learning · ImageNet, ResNet-18 | Top-1 accuracy (%) | Helped |
| Select the top-n frequent codewords from a trained BNN instead of learning the selection end-to-end (ablation) | Sparks · ImageNet, ResNet-18 | Top-1 accuracy (%) | Hurt |
| Select codewords randomly instead of learning the selection end-to-end (ablation) | Sparks · ImageNet, ResNet-18 | Top-1 accuracy (%) | Hurt |
| Select codewords at equal intervals of codebook indices (ablation) | Sparks · ImageNet, ResNet-18 | Top-1 accuracy (%) | Hurt |
| Replenish an overlapping product-quantization codebook with random codewords (ablation) | Product quantization · ImageNet, ResNet-18 | Top-1 accuracy (%) | No clear effect |
| Learn the codeword permutation only during the first 30×10^3 training steps, then fix it (ablation) | Sparks · ImageNet, ResNet-18 | Top-1 accuracy (%) | No clear effect |
| Pretrain the sub-codebook on CIFAR10 and transfer it to ImageNet without further permutation learning (ablation) | Sparks · ImageNet, ResNet-18 | Top-1 accuracy (%): -0.4%, -0.6%, and -1.1% | Hurt |
| Change iteration count k and temperature τ near their default settings (ablation) | PSTE · CIFAR10, ResNet-18 and VGG-small | Top-1 accuracy (%) | No clear effect |
Closest papers by meaning
- An Empirical study of Binary Neural Networks' Optimisation (ICLR 2019 · similarity 0.88)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.87)
- Binarized Neural Networks (NeurIPS 2016 · similarity 0.86)
- Training binary neural networks with real-to-binary convolutions (ICLR 2020 · similarity 0.86)
- BD-Net: Has Depth-Wise Convolution Ever Been Applied in Binary Neural Networks? (AAAI 2026 · similarity 0.86)
- Codebook Features: Sparse and Discrete Interpretability for Neural Networks (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.