Research map / Optimization / Nonconvex and smooth optimization
Efficiently testing local optimality and escaping saddles for ReLU networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use shared extreme-ray tests instead of enumerating all activation regions | SOSP-Check · Two-layer ReLU-like empirical risk | first-order test complexity: O(2^M) to O(M) | Helped |
| Use projected gradient descent for the equality-constrained second-order QP | SOSP-Check · Benign case with no flat extreme rays | QP solution time per update: O(p^2) | Helped |
| Handle flat extreme rays with inequality-constrained QPs | SOSP-Check · Cases with L flat extreme rays | QP solving time complexity: O(p^3 + L^3 2^L) | Mixed |
| Train on larger model/data settings in the reported artificial-data experiments | Adam-trained ReLU networks · Artificial standard-normal data | average flat extreme rays (L): 0 (0) to 37 (0.925) | Mixed |
| Apply the SOSP-Check approach to trained networks with boundary data points | SOSP-Check · Artificial standard-normal data; 280 runs | flat extreme rays: usually zero or very few (maximum was 3) | Helped |
Closest papers by meaning
- The Hidden Convex Optimization Landscape of Regularized Two-Layer ReLU Networks: an Exact Characterization of Optimal Solutions (ICLR 2022 · similarity 0.90)
- Optimal Sets and Solution Paths of ReLU Networks (ICML 2023 · similarity 0.90)
- Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.90)
- Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape (ICLR 2026 · similarity 0.89)
- Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding (ICLR 2025 · similarity 0.88)
- Convex Formulations for Training Two-Layer ReLU Neural Networks (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.