Research map / Optimization / Implicit bias and generalization
Parallel Deep Neural Networks Have Zero Duality Gap
Yifei Wang, Tolga Ergen, Mert Pilanci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use parallel branches with L-th power Frobenius regularization instead of standard architecture with weight decay (deep linear) | deep linear networks, L>=3 · theoretical analysis, vector-output linear networks | duality gap (P vs D) | Improved |
| Stacking standard deep ReLU networks in parallel with modified regularization (sufficient branches) | deep ReLU networks, arbitrary depth · theoretical analysis | duality gap (P vs D) | Improved |
| Standard deep linear network with L>=3 (no parallel branches) | standard deep linear networks · theoretical analysis | duality gap (P vs D) | Hurt |
| Weight decay (l2 regularization) on standard deep linear networks | standard deep linear networks · theoretical analysis | solution rank | No clear effect |
| Rank-1 data matrix for three-layer scalar-output standard ReLU network | three-layer standard ReLU networks · theoretical analysis | duality gap (P vs D) | Improved |
Closest papers by meaning
- Revealing the Structure of Deep Neural Networks via Convex Duality (ICML 2021 · similarity 0.93)
- Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs (ICML 2021 · similarity 0.91)
- Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks (NeurIPS 2023 · similarity 0.90)
- The Hidden Convex Optimization Landscape of Regularized Two-Layer ReLU Networks: an Exact Characterization of Optimal Solutions (ICLR 2022 · similarity 0.89)
- Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.88)
- Convex Formulations for Training Two-Layer ReLU Neural Networks (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
