Research map / Architectures and efficiency / RNNs and normalization
Revealing the Structure of Deep Neural Networks via Convex Duality
Tolga Ergen, Mert Pilanci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use closed-form optimal weights instead of SGD training | Deep ReLU networks · Whitened MNIST and CIFAR-10 | training objective | Improved |
| Use closed-form optimal weights instead of SGD training | Deep ReLU networks · Whitened MNIST and CIFAR-10 | test error | Improved |
| Use closed-form optimal weights instead of full-batch SGD | Two-layer ReLU network with batch normalization · CIFAR-100 | training objective | Improved |
| Use closed-form optimal weights instead of full-batch SGD | Two-layer ReLU network with batch normalization · CIFAR-100 | test accuracy | Improved |
| Add batch normalization between layers | Deep ReLU networks · Arbitrary data | weight alignment characterization | Improved |
| Use one-dimensional rank-one data | Regularized deep ReLU networks · One-dimensional interpolation | spline interpolation | Improved |
| Use whitened inputs and one-hot labels | Deep ReLU networks · Whitened data | availability of closed-form optimal weights | Improved |
Closest papers by meaning
- Parallel Deep Neural Networks Have Zero Duality Gap (ICLR 2023 · similarity 0.93)
- Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs (ICML 2021 · similarity 0.93)
- Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse (ICLR 2025 · similarity 0.89)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.89)
- Exploring The Loss Landscape Of Regularized Neural Networks Via Convex Duality (ICLR 2025 · similarity 0.88)
- Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
