Research map / Learning theory and algorithms / Sample complexity
Analytical Guarantees on Numerical Precision of Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Set BW = BA + 3 (weights get 3 more bits than activations) per the Eq. 12 ratio, versus BW = BA (ablation) | MLP on MNIST · MNIST | minimum activation bits before accuracy degrades: BA reducible to about 3-4 bits vs about 6-7 bits with BW=BA | Helped |
| Use the analytical bound (Theorems 1/2) to choose precision instead of fixed-point simulation | MLP on MNIST · MNIST | test error (%) at chosen precision: (8,8) 1.41 vs float 1.36; (6,6) 1.54 | No clear effect |
| Quantize weights and activations to 4-6 bits total precision with a 3-bit weight surplus | MLP on MNIST · MNIST | computational cost (10^6 FAs): 44.7 (4,7) vs 117 for BinaryNet at similar error | Helped |
| Replace SQ's CNN with a deeper network-in-network style CNN and quantize to 10-bit fixed point | CNN on CIFAR-10 · CIFAR-10 | test error (%): 17.23 vs SQ 25.4 | Helped |
| Reduce precision of weights and activations jointly to 10-12 bits in CNN | CNN on CIFAR-10 · CIFAR-10 | test error (%): 12-bit 17.08 and 10-bit 17.23 vs float 17.02 | No clear effect |
| Set BW=BA+3 for MLP (vs BW=BA) based on EW/EA ratio | CIFAR-10 CNN · CIFAR-10 | precision ratio guidance: log2 sqrt(EW/EA) ~0.29 so BW=BA kept | No clear effect |
| Use the tighter Chernoff-based bound (Theorem 2) rather than Chebyshev-based bound (Theorem 1) for choosing precision (ablation) | MLP on MNIST · MNIST | minimum precision (BA=BW): 6 bits vs 8 bits at pm<=1% | Helped |
Closest papers by meaning
- The Effect of Weight Precision on the Neuron Count in Deep ReLU Networks (ICML 2024 · similarity 0.87)
- Dynamic Back-Substitution in Bound-Propagation-Based Neural Network Verification (AAAI 2025 · similarity 0.86)
- Understanding Certified Training with Interval Bound Propagation (ICLR 2024 · similarity 0.86)
- DeepBern-Nets: Taming the Complexity of Certifying Neural Networks Using Bernstein Polynomial Activations and Precise Bound Propagation (AAAI 2024 · similarity 0.86)
- On the Hardness of Training Deep Neural Networks Discretely (AAAI 2025 · similarity 0.86)
- The Cost of Robustness: Tighter Bounds on Parameter Complexity for Robust Memorization in ReLU Nets (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.