Research map / Architectures and efficiency / RNNs and normalization
Topology of Deep Neural Networks
Gregory Naitzat, Andrey Zhitnikov, Lek-Heng Lim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ReLU instead of tanh activation (ablation) | deep fully connected classifier · D-I, D-II, D-III simulated manifolds | reduction of Betti numbers across layers | Improved |
| use leaky ReLU instead of ReLU activation (ablation) | deep fully connected classifier · simulated manifolds | speed of Betti number reduction | Hurt |
| use tanh with lower floating point precision (larger unit roundoff) | tanh network · topology change | topology-changing ability: delta 2^-54 -> 2^-9 | Improved |
| add depth to constant-width network (ablation) | fifteen-neuron-wide networks · simulated manifolds | fraction of networks trained to high accuracy | Improved |
| shift topological work to final layers when depth is reduced (ablation) | shallower networks · simulated manifolds | layerwise Betti reduction | Mixed |
| add a three-neuron bottleneck layer to ten-layer networks (ablation) | width-15 network · D-I | topology change | Improved |
| use narrow networks (6 neurons) instead of wide (50 neurons) (ablation) | ten-layer network · D-I | speed of topology change | Improved |
| increase width from 15 to 50 neurons (ablation) | ten-layer network · D-II and D-III | topology change | No clear effect |
| reduce width below fifteen neurons (ablation) | ten-layer network · simulated data | percentage of successfully trained networks | Hurt |
| use tanh instead of ReLU or leaky ReLU (ablation) | ten-layer network · D-I | variance of beta_0 decay | Hurt |
Closest papers by meaning
- Low-dimensional topology of deep neural networks (ICML 2026 · similarity 0.91)
- Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity (ICLR 2025 · similarity 0.87)
- Topological obstruction to the training of shallow ReLU neural networks (NeurIPS 2024 · similarity 0.86)
- Geometric Inductive Biases of Deep Networks: The Role of Data and Architecture (ICLR 2025 · similarity 0.86)
- Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework (NeurIPS 2025 · similarity 0.86)
- Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
