Research map / Optimization / Implicit bias and generalization

Learning Hierarchical Polynomials of Multiple Nonlinear Features

ICLR 2025 · Optimization › Implicit bias and generalization · approach: Low-rank and single-index recovery

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Theoretical analysis of how three-layer neural networks learn hierarchical functions f* = g* ∘ p, where p is a vector of multiple quadratic nonlinear features and g* is a polynomial over them.

Problem

Understanding how neural networks learn hierarchical features, specifically learning hierarchical polynomials built from multiple nonlinear (quadratic) features with three-layer networks.

Approach

Studies a broad class of hierarchical targets f* = g* ∘ p with p a vector of multiple quadratic features and g* a polynomial outer function, analyzing learnability by three-layer networks rather than only single-feature settings.

Closest papers by meaning

• Learning Hierarchical Polynomials with Three-Layer Neural Networks (ICLR 2024 · similarity 0.94)
• A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks (ICML 2024 · similarity 0.87)
• Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit (NeurIPS 2024 · similarity 0.87)
• Three Mechanisms of Feature Learning in a Linear Network (ICLR 2025 · similarity 0.86)
• Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws (NeurIPS 2025 · similarity 0.86)
• Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks (NeurIPS 2025 · similarity 0.85)