Research map / Probabilistic and scientific modeling / Variational and generative sampling
Critical initialisation in continuous approximations of binary neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize binary weight means close to ±1, with bias variance near zero | Continuous surrogates for binary neural networks · Deep binary neural networks | trainability | Helped |
| Use the deterministic surrogate instead of the LRT surrogate | Networks with stochastic binary weights and neurons · Arbitrary-depth networks | critical initialization availability | Helped |
| Use the LRT surrogate with continuous tanh neurons | Networks with stochastic binary weights · Arbitrary-depth networks | critical initialization availability | Helped |
| Use the LRT surrogate with binary neurons rather than a surrogate with critical initialization | Networks with stochastic binary weights and neurons · Arbitrary-depth networks | critical initialization availability | Hurt |
| Increase the number of Monte Carlo samples when evaluating the stochastic binary network | Stochastic binary counterpart of the deterministic surrogate · MNIST | training performance | Helped |
| Evaluate with more Monte Carlo samples rather than a single sample | Stochastic binary network · MNIST | classification performance | Helped |
Closest papers by meaning
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.87)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.86)
- Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes (ICLR 2024 · similarity 0.86)
- Sinusoidal Initialization, Time for a New Start (NeurIPS 2025 · similarity 0.86)
- Partially Stochastic Infinitely Deep Bayesian Neural Networks (ICML 2024 · similarity 0.85)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.