Research map / Optimization / Implicit bias and generalization
Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
Arda Sahiner, Tolga Ergen, John M. Pauly, Mert Pilanci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use soft-thresholded SVD under the stated whitened-data and singular-vector conditions | Two-layer ReLU network · Whitened CIFAR-10 and CIFAR-100 | training objective | No clear effect |
| Use soft-thresholded SVD instead of SGD | Two-layer ReLU network · Whitened CIFAR-10 | solve time: 0.018 seconds | Improved |
| Use soft-thresholded SVD instead of SGD | Two-layer ReLU network · Whitened CIFAR-100 | solve time: 0.36 seconds | Improved |
| Increase hidden units from 10 to 50 | SGD-trained two-layer ReLU network · Synthetic Gaussian inputs with generated labels | training loss relative to global optimum | Improved |
| Replace ReLU activation with linear activation (ablation) | Two-layer network on whitened CIFAR-10 continuous labels · CIFAR-10 test set | test loss | Hurt |
| Use ReLU rather than the zero network at β=10^{-2} (ablation) | Two-layer network on whitened CIFAR-10 continuous labels · CIFAR-10 test set | test loss: 125383 -> 63038 | Improved |
| Use the copositive relaxation | Vector-output ReLU network · Spiral classification | training loss | No clear effect |
| Use the copositive relaxation instead of SGD | Vector-output ReLU network · Whitened MNIST and CIFAR-10 image fitting | training objective | No clear effect |
Closest papers by meaning
- The Hidden Convex Optimization Landscape of Regularized Two-Layer ReLU Networks: an Exact Characterization of Optimal Solutions (ICLR 2022 · similarity 0.92)
- Optimal Sets and Solution Paths of ReLU Networks (ICML 2023 · similarity 0.91)
- Convex Formulations for Training Two-Layer ReLU Neural Networks (ICLR 2025 · similarity 0.91)
- Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.90)
- Global Minimizers of $\ell^p$-Regularized Objectives Yield the Sparsest ReLU Neural Networks (NeurIPS 2025 · similarity 0.88)
- Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
