Research map / Optimization / Implicit bias and generalization
Globally Optimal Training of Neural Networks with Threshold Activation Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| convex Lasso formulation instead of non-convex STE training | two-layer threshold networks · UCI/CIFAR-10/MNIST (13 datasets) | test accuracy: best accuracy on 9 of 13 datasets vs 2 of 13 for STE | Mixed |
| convex Lasso formulation instead of Nonconvex-STE | two-layer threshold networks · CIFAR-10 | training time (s): 8.9 vs 83.5 s | Helped |
| complete arrangements simplified convex formulation (delta form) when m >= n+2 | two-layer threshold networks · synthetic | training objective: convex programs achieve lower objective than all non-convex heuristics | Helped |
| Convex-Lasso with subsampled random arrangements instead of full enumeration | convex Lasso formulation · synthetic | training objective | No clear effect |
Closest papers by meaning
- Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs (ICML 2021 · similarity 0.87)
- DEEP NEURAL NETWORK INITIALIZATION WITH SPARSITY INDUCING ACTIVATIONS (ICLR 2024 · similarity 0.87)
- Deep Learning as a Mixed Convex-Combinatorial Optimization Problem (ICLR 2018 · similarity 0.87)
- Gradient Descent Provably Optimizes Over-parameterized Neural Networks (ICLR 2019 · similarity 0.87)
- Rethinking Neural Nonlinearity as Gating (NeurIPS 2026 · similarity 0.87)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.