Research map / Learning theory and algorithms / Kernels and regression theory
Deep Information Propagation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Keep network depth at or below approximately six correlation depth-scales | Random fully connected networks · MNIST and CIFAR10 | trainability: L\lesssim 6\xi_c | Helped |
| Initialize closer to the order-to-chaos critical point | Random fully connected networks · MNIST and CIFAR10 | training speed | Helped |
| Add dropout | Random fully connected networks · MNIST | maximum trainable depth: about L = 100 | Hurt |
| Increase dropout rate | Random fully connected networks · Signal propagation | correlation depth scale ξ_c | Hurt |
| Move from the ordered phase to the chaotic phase | Random neural networks · Gradient backpropagation | gradient magnitude | Mixed |
Closest papers by meaning
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.88)
- Exponential expressivity in deep neural networks through transient chaos (NeurIPS 2016 · similarity 0.88)
- Uniform-in-Time Weak Propagation of Chaos in Shallow Neural Networks (NeurIPS 2026 · similarity 0.86)
- Generalization of Scaled Deep ResNets in the Mean-Field Regime (ICLR 2024 · similarity 0.86)
- Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit (ICLR 2024 · similarity 0.85)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.