Research map / Optimization / Implicit bias and generalization

The Directionality of Optimization Trajectories in Neural Networks

ICLR 2025 · Optimization › Implicit bias and generalization · approach: Edge-of-stability dynamics

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies the directionality of gradient descent optimization trajectories in neural networks as a lens on implicit bias, beyond parameter norms and curvature.

Problem

Implicit bias and regularity in neural network optimization are usually studied through parameter norms or landscape curvature, overlooking the trajectory that gradient descent takes to reach solutions.

Approach

Introduces a notion of directionality of optimization trajectories, capturing how gradient descent navigates the landscape toward a solution, as a property distinct from norms and curvature.

Closest papers by meaning

• No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.90)
• Understanding Optimization in Deep Learning with Central Flows (ICLR 2025 · similarity 0.89)
• Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning (NeurIPS 2026 · similarity 0.88)
• The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions (ICML 2025 · similarity 0.88)
• Gradient Descent as Loss Landscape Navigation: a Normative Framework for Deriving Learning Rules (NeurIPS 2025 · similarity 0.87)
• Directional Smoothness and Gradient Methods: Convergence and Adaptivity (NeurIPS 2024 · similarity 0.87)