Research map / Architectures and efficiency / RNNs and normalization
Complexity of Linear Regions in Deep Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use random initialization with bounded pre-activation gradient | Piecewise-linear networks · Any fixed 1D line segment | number of linear regions | Helped |
| Increase depth while keeping the total number of neurons fixed | Randomly initialized networks · Any fixed 1D curve | number of linear regions | No clear effect |
| Use random initialization under the paper's stated assumptions | Piecewise-linear networks · Bounded input set | linear-region boundary volume density | Helped |
| Use random initialization under the paper's stated assumptions | Piecewise-linear networks · Uniform inputs in a compact set | distance to nearest region boundary | Helped |
| Train the initialized networks on MNIST | Fully connected ReLU networks · MNIST, lines through the origin and sampled training examples | number of linear regions along a line | No clear effect |
| Train the initialized networks on MNIST | Fully connected ReLU networks · MNIST-like random input points | distance to nearest region boundary | Mixed |
| Train a depth-4, width-16 network | Fully connected ReLU network · MNIST, sampled input points | distribution of distances to nearest region boundary | No clear effect |
| Train the network | Fully connected ReLU network, depth 3 and width 64 · MNIST, 2D plane through input space | linear regions in the plane | Mixed |
Closest papers by meaning
- Empirical Studies on the Properties of Linear Regions in Deep Neural Networks (ICLR 2020 · similarity 0.92)
- Bounding and Counting Linear Regions of Deep Neural Networks (ICML 2018 · similarity 0.92)
- On the Number of Linear Regions of Convolutional Neural Networks (ICML 2020 · similarity 0.92)
- The Computational Complexity of Counting Linear Regions in ReLU Neural Networks (NeurIPS 2025 · similarity 0.91)
- On the Local Complexity of Linear Regions in Deep ReLU Networks (ICML 2025 · similarity 0.88)
- Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.