AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Light Differentiable Logic Gate Networks

Lukas Rüttgers, Till Aczel, Andreas Plesner, Roger Wattenhofer

ICLR 2026 · Architectures and efficiency › RNNs and normalization · approach: Efficient network compression and training · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace softmax over 16 gates by input-wise parametrization (IWP)DLGN · CIFAR-10 M DLGNmodel size: 4x smallerImproved
Use IWP instead of OPDLGN 20-fold depth · CIFAR-100training steps to best OP accuracy: 222000 -> 26000Improved
Use IWP instead of OPDLGN 80 layers, batch size 1 · runtimebackward pass time: 1.86x speedupImproved
Use IWP instead of OP at large batch sizeDLGN · training efficiencyrelative speedupNo clear effect
Use IWP with RI instead of OP with RI in deeper networks (ablation)CDLGN 5-fold depth · CIFAR-100 discretizedtest accuracy: more than 1.3 times betterImproved
Use IWP alone without residual initialization (ablation)DLGN 40 layers · gradient normgradient norm: 10^{-34} -> 10^{-16}No clear effect
Add residual initialization (RI) to IWP (ablation)deep DLGN · CIFAR-100accuracy and gradient stabilityImproved
Use AND-OR initialization instead of RI (ablation)IWP DLGN · CIFAR-100accuracy and discretization errorHurt
Lower initialization variance of IWP with RI (ablation)deeper IWP DLGN · CIFAR-100convergence and test accuracyNo clear effect
Increase depth with IWP (ablation)DLGN · CIFAR-100test accuracyNo clear effect
Use sinusoidal instead of sigmoid gate estimator (ablation)IWP · CIFAR-100accuracyImproved
Use linear straight-through estimator instead of sinusoidal (ablation)IWP DLGN · runtimeforward / backward time: forward -22%, backward -4%Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy