Research map / Architectures and efficiency / Quantization
BOLD: Boolean Logic Deep Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Boolean Logic Deep Learning (BOLD), a Boolean-variation calculus that trains networks with native Boolean weights and activations directly in the Boolean domain, without latent full-precision weights or gradient descent.
Problem
Binarized/quantized neural network training relies on full-precision latent weights and approximate gradients through sign-like discretization, causing accuracy loss and high training cost; no scalable algorithm exists for natively training deep binary models.
Approach
Introduces Boolean variation (a discrete analogue of derivative with a chain rule via XNOR) and a Boolean optimizer that flips weights when the accumulated Boolean optimization signal agrees with the weight's sign, so training runs entirely in Boolean logic with no latent FP weights.
Key results
- VGG-Small with BN on CIFAR-10 reaches 92.37% accuracy vs 93.80% FP baseline, using under 5% of FP energy.
- ResNet18-based BOLD reaches 66.9% (standard) and 70.0% (4x wider, KD with ResNet34) top-1 on ImageNet, with 24.45% energy relative to the FP baseline for the large model.
- Boolean DeepLabV3 reaches 67.4% mIoU on Cityscapes vs 70.7% FP and above Binary DAD-Net (58.1%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add BN (ablation) | BOLD VGG-small · CIFAR10 | accuracy: 90.29 -> 92.37 | Helped |
| use native Boolean training instead of BinaryNet | ResNet18 · ImageNet | accuracy: 42.2 -> 51.8 | Helped |
| use BOLD instead of the strongest listed KD baseline | ResNet18 with ResNet34 teacher · ImageNet | accuracy: 68.4 -> 70.0 | Helped |
| add BN (ablation) | BOLD ResNet18 · ImageNet | accuracy: 70.0 -> 65.9 | Hurt |
| use BOLD instead of Binary DAD-Net | DeepLabV3 · Cityscapes | mIoU: 58.1 -> 67.4 | Helped |
| use BOLD instead of the full-precision baseline | DeepLabV3 · Cityscapes | mIoU: 70.7 -> 67.4 | Hurt |
| use BOLD instead of the full-precision baseline | DeepLabV3 · Pascal VOC 2012 | mIoU: 72.1 -> 67.3 | Hurt |
| use BOLD instead of Bit | BERT · GLUE average | accuracy: 71.0 -> 70.9 | No clear effect |
| fine-tune a CIFAR100-trained BOLD model on CIFAR10 rather than train from scratch | VGG-small · CIFAR10 | accuracy: 90.29 -> 92.09 | Helped |
| fine-tune a CIFAR10-trained BOLD model on CIFAR100 rather than train from scratch | VGG-small · CIFAR100 | accuracy: 68.43 -> 68.37 | No clear effect |
| use BOLD instead of full EDSR | EDSR · Set5, ×2 | PSNR (dB): 38.11 -> 37.42 | Hurt |
Surprising findings
- Boolean VGG-Small without BatchNorm (90.29%) closely matches BinaryConnect despite using 1-bit activations and weights.
- On BERT GLUE, the Boolean model averages 70.9%, comparable to BiT (71.0%) and above BinaryBERT and BiBERT, despite natively using Boolean weights.
Limitations
- Experiments run on GPUs designed for real arithmetic; energy is estimated analytically, not measured on native Boolean hardware.
- No universal approximation guarantee for Boolean neural networks.
- Large accuracy gaps remain on some tasks (e.g., ImageNet ResNet18 standard model 51.8% vs 69.7% FP; BERT GLUE drops from 83.9%).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet Cityscapes PASCAL VOC 2012 DIV2K GLUE
Benchmarks: CIFAR-10 CIFAR-100 ImageNet Set5 Set14 BSD100 Urban100 DIV2K Cityscapes PASCAL VOC 2012 GLUE
Metrics: accuracy PSNR mIoU energy consumption (% of FP baseline)
Baselines: BinaryConnect XNOR-Net BinaryNet Bi-Real Net ReActNet BNext PokeBNN Real2Binary MeliusNet BinaryBERT BiBERT BiT
Builds on: BinaryConnect BinaryNet XNOR-Net Bi-Real Net BiT EDSR
Closest papers by meaning
- Verifying Properties of Binarized Deep Neural Networks (AAAI 2018 · similarity 0.88)
- Not All Bits have Equal Value: Heterogeneous Precisions via Trainable Noise (NeurIPS 2022 · similarity 0.86)
- Mind the Gap: Removing the Discretization Gap in Differentiable Logic Gate Networks (NeurIPS 2025 · similarity 0.85)
- Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling (NeurIPS 2025 · similarity 0.84)
- Convolutional Differentiable Logic Gate Networks (NeurIPS 2024 · similarity 0.84)
- Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.