Research map / Learning theory and algorithms / Kernels and regression theory
Deep Networks and the Multiple Manifold Problem
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase network depth to at least a constant multiple of inverse manifold separation | Deep fully-connected ReLU network on two coaxial circles · Two coaxial circles | certificate existence: L \geq C_1 \Delta\inv | Helped |
| Use sufficient depth, width, samples, and a small step size, assuming a suitably small certificate exists | Gradient descent on a deep fully-connected ReLU network · Two 1D manifolds | separation guarantee: L \geq \poly(\kappa,C_{\rho}, \log(\adim)); n \geq \poly(L,\log(L \adim)); N \geq \poly(L) | Helped |
| Scale width at least linearly with depth | Deep fully-connected ReLU network · d-dimensional submanifold of the unit sphere | uniform NTK concentration: n~\geq~C_1 L \mdim^4 \log^4(C_{\manifold} \adim L) | Helped |
| Increase depth | Deep fully-connected ReLU network · Multiple manifold classification | kernel localization for fitting | Helped |
| Increase width | Deep fully-connected ReLU network · Multiple manifold classification | concentration of the initial NTK | Helped |
| Use more training samples | Deep fully-connected ReLU network with fixed architecture · Multiple manifold classification | fitting | No clear effect |
Closest papers by meaning
- A Statistical Recurrent Model on the Manifold of Symmetric Positive Definite Matrices (NeurIPS 2018 · similarity 0.90)
- Hardness of Learning Neural Networks under the Manifold Hypothesis (NeurIPS 2024 · similarity 0.88)
- Building Transformation Layers for Riemannian Neural Networks (NeurIPS 2026 · similarity 0.88)
- Matrix Manifold Neural Networks++ (ICLR 2024 · similarity 0.87)
- Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves (NeurIPS 2026 · similarity 0.87)
- A Manifold Perspective on the Statistical Generalization of Graph Neural Networks (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.