Research map / Architectures and efficiency / RNNs and normalization
Light Differentiable Logic Gate Networks
Lukas Rüttgers, Till Aczel, Andreas Plesner, Roger Wattenhofer
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace softmax over 16 gates by input-wise parametrization (IWP) | DLGN · CIFAR-10 M DLGN | model size: 4x smaller | Improved |
| Use IWP instead of OP | DLGN 20-fold depth · CIFAR-100 | training steps to best OP accuracy: 222000 -> 26000 | Improved |
| Use IWP instead of OP | DLGN 80 layers, batch size 1 · runtime | backward pass time: 1.86x speedup | Improved |
| Use IWP instead of OP at large batch size | DLGN · training efficiency | relative speedup | No clear effect |
| Use IWP with RI instead of OP with RI in deeper networks (ablation) | CDLGN 5-fold depth · CIFAR-100 discretized | test accuracy: more than 1.3 times better | Improved |
| Use IWP alone without residual initialization (ablation) | DLGN 40 layers · gradient norm | gradient norm: 10^{-34} -> 10^{-16} | No clear effect |
| Add residual initialization (RI) to IWP (ablation) | deep DLGN · CIFAR-100 | accuracy and gradient stability | Improved |
| Use AND-OR initialization instead of RI (ablation) | IWP DLGN · CIFAR-100 | accuracy and discretization error | Hurt |
| Lower initialization variance of IWP with RI (ablation) | deeper IWP DLGN · CIFAR-100 | convergence and test accuracy | No clear effect |
| Increase depth with IWP (ablation) | DLGN · CIFAR-100 | test accuracy | No clear effect |
| Use sinusoidal instead of sigmoid gate estimator (ablation) | IWP · CIFAR-100 | accuracy | Improved |
| Use linear straight-through estimator instead of sinusoidal (ablation) | IWP DLGN · runtime | forward / backward time: forward -22%, backward -4% | Mixed |
Closest papers by meaning
- Mind the Gap: Removing the Discretization Gap in Differentiable Logic Gate Networks (NeurIPS 2025 · similarity 0.93)
- Convolutional Differentiable Logic Gate Networks (NeurIPS 2024 · similarity 0.91)
- LogicMP: A Neuro-symbolic Approach for Encoding First-order Logic Constraints (ICLR 2024 · similarity 0.86)
- Towards Next-Generation Logic Synthesis: A Scalable Neural Circuit Generation Framework (NeurIPS 2024 · similarity 0.84)
- Analog Foundation Models (NeurIPS 2025 · similarity 0.84)
- BOLD: Boolean Logic Deep Learning (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
