Research map / Architectures and efficiency / Quantization
BiPer: Binary Neural Networks using a Periodic Function
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BiPer, a binary neural network method that binarizes weights with a square-wave (sign of sine) in the forward pass and uses a sine surrogate gradient in the backward pass, with frequency controlling quantization error.
Problem
Binary neural networks suffer accuracy degradation because the sign function has zero gradient almost everywhere, and STE-style smooth surrogates create forward/backward mismatch that destabilizes gradients.
Approach
Use a binary periodic square wave (sign of sin(ω0 w)) for binarization in the forward pass and the derivative of the sine function with the same period as a differentiable surrogate in the backward pass; the frequency ω0 controls quantization error and is used to initialize weights in a two-stage training scheme.
Key results
- On CIFAR-10 ResNet-18, BiPer reaches 93.75% Top-1 with 1/1-bit weights and activations, about 1% above ReCU (92.8%).
- On ImageNet ResNet-18 1W/1A, BiPer reaches 61.4% Top-1 and 83.14% Top-5, versus 61.0% and 82.6% for ReCU.
- On ImageNet ResNet-34 1W/1A, BiPer reaches 65.73% Top-1 and 86.39% Top-5, improving on ReCU by about 0.63% Top-1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use periodic binarization rather than sign binarization in the second training stage (ablation) | BiPer two-stage training · CIFAR-10, ResNet-18/20 | Precision: 93.01 -> 93.93 | Helped |
| Vary the periodic-function frequency ω0 (ablation) | BiPer · CIFAR-10, ResNet-18/20 | Top-1 precision | Mixed |
| Choose an intermediate frequency ω0 = 20 (ablation) | BiPer · CIFAR-10, ResNet-18/20 | Top-1 precision | Helped |
| Use a higher frequency to reduce weight quantization error (ablation) | BiPer Stage 1 · CIFAR-10, ResNet-18/20 | Top-1 precision | Hurt |
| Binarize weights and keep activations real instead of binarizing both (ablation) | BiPer · CIFAR-10, ResNet-20 | Top-1 accuracy: 91.20% vs 91.20% | No clear effect |
Surprising findings
- Lowest weight quantization error in stage 1 does not yield the best final binary model; an intermediate frequency (ω0=20) gave the best balance.
- In some settings (ω0=20, 30) the final binary BiPer model exceeded the accuracy of the real-valued initialization model.
Limitations
- Gradient is proportional to the frequency, so increasing frequency to reduce quantization error causes divergent gradients.
- Evaluated only on image classification with CIFAR-10 and ImageNet; VGG-Small results are only competitive, not best.
Experimental setup
Datasets: CIFAR-10 ImageNet ILSVRC-2012
Benchmarks: CIFAR-10 ImageNet
Metrics: Top-1 accuracy Top-5 accuracy quantization error
Baselines: XNOR-Net BNN DoReFa DSQ SLB IR-Net RBNN ReCU ReSTE MST DIR-Net LCR-BNN
Builds on: straight-through estimator Bi-Real Net ReSTE ReCU IR-Net
Closest papers by meaning
- Understanding Neural Network Binarization with Forward and Backward Proximal Quantizers (NeurIPS 2023 · similarity 0.88)
- Sparsity-Inducing Binarized Neural Networks (AAAI 2020 · similarity 0.87)
- BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training (ICLR 2026 · similarity 0.87)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.85)
- Fast and Slow Gradient Approximation for Binary Neural Network Optimization (AAAI 2025 · similarity 0.85)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.