Research map / Optimization / Implicit bias and generalization
Directional convergence and alignment in deep learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Require positive homogeneity, definability, and initial risk below the loss at zero | Gradient flow for deep classification networks · Binary classification with logistic or exponential loss | directional convergence of normalized weights | Helped |
| Require locally Lipschitz gradients in addition to the main assumptions | Gradient flow for deep classification networks · Binary classification with logistic or exponential loss | alignment between weights and negative risk gradient | Helped |
| Use homogeneous networks with directional convergence and gradient alignment | Deep linear networks · Deep linear classification | convergence to the linear max-margin predictor | Helped |
| Add the covering condition to the finite-width squared-ReLU network setting | Finite-width 2-homogeneous squared-ReLU network · Binary classification with bounded-norm inputs | global margin maximization | Helped |
| Use an unmodified DenseNet rather than a homogeneous network | DenseNet · CIFAR-10 | margin convergence | Helped |
| Use SGD instead of gradient descent | Squared-ReLU network and CIFAR convolutional networks · Synthetic data and CIFAR | directional convergence behavior | No clear effect |
Closest papers by meaning
- On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths (ICML 2021 · similarity 0.90)
- Gradient Descent Maximizes the Margin of Homogeneous Neural Networks (ICLR 2020 · similarity 0.90)
- Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks (ICML 2025 · similarity 0.89)
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.89)
- Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning (NeurIPS 2026 · similarity 0.88)
- Embedding Principle of Homogeneous Neural Network for Classification Problem (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.