Research map / Architectures and efficiency / RNNs and normalization
Monotone Learning with Rectified Wire Networks
Veit Elser, Dan Schmidt, Jonathan Yedidia
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use strict zero-output termination instead of ultra-conservative termination (ablation) | SDA on 216-edge network · 3-variable Boolean f> (majority) | training success rate: 100% -> 76% | Hurt |
| use strict zero-output termination instead of ultra-conservative termination (ablation) | SDA on 216-edge network · 3-variable Boolean f+ (parity) | training success rate: 100% -> 22% | Hurt |
| use strict zero-output termination instead of ultra-conservative termination (ablation) | SDA on 216-edge network · 3-variable Boolean f1 | total SDA iterations: about 140, or about 15 times the number needed by the ultra-conservative rule | Hurt |
| reduce weight multiplier q from 1 to 0.5 under the zero-output rule (ablation) | SDA with strict zero-output rule · 3-variable Boolean f1, f>, f+ | training success rate: 76% -> 100% on f>; 22% -> 100% on f+ | Improved |
| use zero-output SDA instead of ultra-conservative SDA (ablation) | SDA on 16-edge network · 2-variable Boolean functions | training success: 6 <= #iter <= 9, #err=4 | No clear effect |
| increase function complexity from f1 to majority to parity | SDA on 216-edge network · 3-variable Boolean functions | #err and #iter: ranking f1 < f> < f+ | Hurt |
Closest papers by meaning
- Expressiveness of Rectifier Networks (ICML 2016 · similarity 0.87)
- Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes (NeurIPS 2024 · similarity 0.85)
- On the Hardness of Training Deep Neural Networks Discretely (AAAI 2025 · similarity 0.85)
- Smooth Min-Max Monotonic Networks (ICML 2024 · similarity 0.85)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.84)
- RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
