Research map / Learning theory and algorithms / Combinatorial and search optimization
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the exact convex program instead of GD | Two-layer ReLU network · 2D synthetic dataset | training objective: 0.0032 -> 0.0026 | Helped |
| Approximate the convex program with sampled sign patterns instead of using GD | Two-layer ReLU network · 2D synthetic dataset | training objective: 0.0032 -> 0.0028 | Helped |
| Use Alg1's sampled sign patterns rather than GD | Two-layer ReLU network · CIFAR-10 binary classification subset | training objective: 0.0042 -> 0.0022 | Helped |
| Use Alg1's sampled sign patterns rather than GD | Two-layer ReLU network · CIFAR-10 binary classification subset | test accuracy: 62.75 -> 66.80 | Helped |
| Use initialization-derived sign patterns (Alg2) rather than Alg1's sampled patterns | Two-layer ReLU network · CIFAR-10 binary classification subset | training objective: 0.0022 -> 0.0032 | Hurt |
| Use initialization-derived sign patterns (Alg2) rather than Alg1's sampled patterns | Two-layer ReLU network · CIFAR-10 binary classification subset | test accuracy: 66.80 -> 60.15 | Hurt |
| Use heuristically adapted initial sign patterns (Alg3) rather than Alg1's sampled patterns | Two-layer ReLU network · CIFAR-10 binary classification subset | training objective: 0.0022 -> 0.0032 | Hurt |
| Use heuristically adapted initial sign patterns (Alg3) rather than Alg1's sampled patterns | Two-layer ReLU network · CIFAR-10 binary classification subset | test accuracy: 66.80 -> 60.20 | Hurt |
| Use the convex ℓ1 formulation rather than GD | Two-layer linear CNN · CIFAR-10 subset | training objective: 0.804 -> 0.803 | Helped |
Closest papers by meaning
- Implicit Convex Regularizers of CNN Architectures: Convex Optimization of Two- and Three-Layer Networks in Polynomial Time (ICLR 2021 · similarity 0.93)
- Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions (ICML 2022 · similarity 0.92)
- Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.90)
- Convex Formulations for Training Two-Layer ReLU Neural Networks (ICLR 2025 · similarity 0.89)
- Global Minimizers of $\ell^p$-Regularized Objectives Yield the Sparsest ReLU Neural Networks (NeurIPS 2025 · similarity 0.87)
- Exploring The Loss Landscape Of Regularized Neural Networks Via Convex Duality (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.