Research map / Architectures and efficiency / RNNs and normalization
Can DBNNs Robust to Environmental Noise for Resource-constrained Scenarios?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an L1,∞-norm constraint on binary weights and scaling factors to improve environmental-noise robustness of deep binary neural networks at inference, with a tighter theoretical upper bound than LCR.
Problem
Deep binary neural networks (DBNNs) deployed on resource-constrained edge devices are vulnerable to environmental noise (e.g., movement artifacts, device noise) during inference, and prior robustness methods are costly or lack formal noise bounds.
Approach
Attributes DBNN noise vulnerability to binary weights and scaling factors, derives a closed-form perturbation upper bound via the L1,∞ norm, and adds an L1,∞-norm regularization term on binarized weights to the cross-entropy loss, avoiding the extra cost of the LCR spectral-norm approximation.
Key results
- On CIFAR-100 with ResNet18, the method improves IRNet robustness accuracy from 65.24% to 70.04% under noise (up to 4.8 points gain across settings).
- On Brain tumor MRI, the method yields 3-5% robustness gains over React, DoReFa and CycleBNN DBNNs (e.g., ReAct ResNet18 76.77 to 83.87).
- Training cost on ImageNet is reduced by 16% relative to LCR (56:08 vs 66:37 per epoch).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the L1,∞-norm constraint to DoReFa | DoReFa with ResNet18 · CIFAR-10 with noise | test accuracy: 91.55 -> 92.27 | Helped |
| Add the L1,∞-norm constraint to BiReal | BiReal with ResNet18 · CIFAR-10 with noise | test accuracy: 91.20 -> 93.21 | Helped |
| Add the L1,∞-norm constraint to ReAct | ReAct with ResNet18 · CIFAR-10 with noise | test accuracy: 91.40 -> 93.04 | Helped |
| Add the L1,∞-norm constraint to IRNet | IRNet with ResNet18 · CIFAR-10 with noise | test accuracy: 91.41 -> 93.64 | Helped |
| Add the L1,∞-norm constraint to IRNet | IRNet with ResNet34 · CIFAR-10 with noise | test accuracy: 87.88 -> 88.31 | Helped |
| Add the L1,∞-norm constraint to DoReFa | DoReFa with ResNet18 · CIFAR-100 with noise | test accuracy: 65.15 -> 67.21 | Helped |
| Add the L1,∞-norm constraint to BiReal | BiReal with ResNet18 · CIFAR-100 with noise | test accuracy: 65.35 -> 68.84 | Helped |
| Add the L1,∞-norm constraint to ReAct | ReAct with ResNet18 · CIFAR-100 with noise | test accuracy: 66.01 -> 68.26 | Helped |
| Add the L1,∞-norm constraint to IRNet | IRNet with ResNet18 · CIFAR-100 with noise | test accuracy: 65.24 -> 70.04 | Helped |
| Add the L1,∞-norm constraint to the binary convolution | IRConv, BirealConv, and AdaBinConv · Bio-electricity with noise | test accuracy: 87.21 -> 88.36; 88.66 -> 89.51; 93.13 -> 93.72 | Helped |
| Add the L1,∞-norm constraint to ReAct | ReAct with ResNet18 · Brain tumor MRI with noise | test accuracy: 76.77 -> 83.87 | Helped |
| Add the L1,∞-norm constraint to DoReFa | DoReFa with ResNet18 · Brain tumor MRI with noise | test accuracy: 84.84 -> 86.91 | Helped |
Surprising findings
- Reports that the DBNN robustness vulnerability stems from binary weights and scaling factors rather than activations alone.
Limitations
- Theoretical derivation omits residual structure analysis for simplicity, with residual bounds deferred to the appendix.
- Robustness is evaluated only under the authors' chosen noise perturbation ranges (L-infinity perturbations and SNR near 50%), not on a standardized noise benchmark.
- Improvements on some settings are small (e.g., about 1% on bio-electrical task).
Experimental setup
Datasets: bio-electricity series classification dataset (Roeder, 2022) CIFAR-10 CIFAR-100 ImageNet Brain tumor MRI (Nickparvar, 2021)
Benchmarks: CIFAR-10 CIFAR-100 ImageNet bio-electricity series classification Brain tumor MRI
Metrics: test accuracy with noise training time per epoch test time per epoch
Baselines: LCR DoReFa-Net Bi-Real Net ReActNet IR-Net AdaBin CycleBNN
Builds on: LCR ReActNet Bi-Real Net DoReFa-Net IR-Net spectral normalization
Closest papers by meaning
- Enhancing Noise-Robust Losses for Large-Scale Noisy Data Learning (AAAI 2025 · similarity 0.84)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.84)
- Understanding Certified Training with Interval Bound Propagation (ICLR 2024 · similarity 0.83)
- Benign overfitting in leaky ReLU networks with moderate input dimension (NeurIPS 2024 · similarity 0.83)
- Ditch the Denoiser: Emergence of Noise Robustness in Self-Supervised Learning from Data Curriculum (NeurIPS 2025 · similarity 0.83)
- Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.