Research map / Point clouds / Point cloud classification
Semi-flat minima and saddle points by embedding neural networks to overparameterization
Kenji Fukumizu, Shoichiro Yamaguchi, Yoh-ichi Mototake, Mirai Tanaka
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add inactive units to a ReLU network | A wider ReLU network embedding a narrower local minimum · Training error landscape | minimum status | Improved |
| Add inactive units to a ReLU network | A wider ReLU network · Training error landscape | flat directions | Improved |
| Replace ReLU with a smooth activation for inactive-unit embedding | A wider network embedding a zero-training-error solution · Training error landscape | flat directions | Hurt |
| Replicate a unit in a smooth network | A wider smooth network · Training error landscape | minimum status | Hurt |
| Replicate a unit in a ReLU network | A wider ReLU network · Training error landscape | minimum status | Hurt |
| Use inactive units without ensuring their outgoing weights are zero (ablation) | Smooth-network embedding · Training error landscape | stationarity | Hurt |
| Use inactive propagation without also making surplus units inactive (ablation) | Smooth-network embedding · Training error landscape | stationarity | Hurt |
| Add inactive units to a smooth network instead of replicating a unit (ablation) | Embedding a minimum in a wider smooth network · Training error landscape | minimum status | Mixed |
| Increase surplus-unit count | ReLU versus tanh networks fitted with inactive units · Generalization error | generalization error trend | Mixed |
| Use ReLU rather than smooth activation for zero-error inactive-unit embeddings | Overparameterized networks with the same number of surplus units · Generalization error bound | PAC-Bayes upper bound | Improved |
Closest papers by meaning
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.88)
- Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization (NeurIPS 2023 · similarity 0.88)
- Bounds on Over-Parameterization for Guaranteed Existence of Descent Paths in Shallow ReLU Networks (ICLR 2020 · similarity 0.88)
- Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding (ICLR 2025 · similarity 0.88)
- Simplicity Bias and Optimization Threshold in Two-Layer ReLU Networks (ICML 2025 · similarity 0.87)
- Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
