Research map / Optimization / Implicit bias and generalization

Scaling Convex Neural Networks with Burer-Monteiro Factorization

ICLR 2024 · Optimization › Implicit bias and generalization · approach: Shallow ReLU network convergence

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Scales convex reformulations of two-layer neural networks by applying Burer-Monteiro factorization to the nuclear-norm-regularized convex program, enabling training at moderate scales.

Problem

Training convex reformulations of two-layer neural networks requires computing an induced low-rank (nuclear norm) regularizer that becomes prohibitively expensive at moderate scales.

Approach

Apply Burer-Monteiro factorization to the convex program so the low-rank-inducing regularizer is replaced by a factored, non-convex but tractable parameterization, allowing convex neural network training to scale.

Experimental setup

Builds on: convex reformulations of two-layer neural networks Burer-Monteiro factorization

Closest papers by meaning

• Smooth Min-Max Monotonic Networks (ICML 2024 · similarity 0.86)
• A Universal Class of Sharpness-Aware Minimization Algorithms (ICML 2024 · similarity 0.86)
• Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.86)
• Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries (ICLR 2025 · similarity 0.86)
• Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.85)
• BPQP: A Differentiable Convex Optimization Framework for Efficient End-to-End Learning (NeurIPS 2024 · similarity 0.85)