Research map / Architectures and efficiency / RNNs and normalization
BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| initialize the binary network by decoupling a pretrained ternary model, then fine-tune | VGG-7 · CIFAR-10 | accuracy: 89.07% -> 90.44% | Helped |
| pretrain with a smaller coupled ternary model (ablation) | VGG-7 · CIFAR-10 | accuracy: 89.07% -> 89.69% | Helped |
| remove ternary pretraining and train the decoupled binary network from scratch (ablation) | BinaryDuo · CIFAR-10, VGG-7 | accuracy: 90.44% -> 88.93% | Hurt |
| replace ternary pretraining with 2-bit pretraining before decoupling (ablation) | BinaryDuo · CIFAR-10, VGG-7 | accuracy: 90.44% -> 89.61% | Hurt |
| use a quarter-sized ternary pretrained model to preserve the baseline architecture after decoupling (ablation) | BinaryDuo-Q · CIFAR-10, VGG-7 | accuracy: 90.44% -> 89.72% | Hurt |
| use BinaryDuo training instead of the strongest listed AlexNet baseline | AlexNet · ImageNet validation | Top-1 accuracy: 48.3 -> 52.7 | Helped |
| use BinaryDuo instead of CBCN | ResNet-18 · ImageNet validation | Top-1 accuracy: 61.4 -> 60.4 | Hurt |
| add an additional shortcut to BinaryDuo (ablation) | ResNet-18 · ImageNet validation | Top-1 accuracy: 60.4 -> 60.9 | Helped |
| use ternary rather than binary activations during gradient-mismatch analysis | binary activation networks · toy feed-forward network with 3 hidden layers | cosine similarity between coarse gradient and CDG | Helped |
| replace conventional STE with sophisticated STEs (ablation) | binary activation networks · toy feed-forward network with 3 hidden layers | cosine similarity between coarse gradient and CDG | No clear effect |
Closest papers by meaning
- Improving Accuracy of Binary Neural Networks Using Unbalanced Activation Distribution (CVPR 2021 · similarity 0.88)
- BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training (ICLR 2026 · similarity 0.87)
- Fast and Slow Gradient Approximation for Binary Neural Network Optimization (AAAI 2025 · similarity 0.86)
- BiPer: Binary Neural Networks using a Periodic Function (CVPR 2024 · similarity 0.85)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.85)
- A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.