Research map / Architectures and efficiency / RNNs and normalization
An Exact Poly-Time Membership-Queries Algorithm for Extracting a Three-Layer ReLU Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Recover neurons from critical points and reconstruct their local affine transformations | Depth-two ReLU networks · δ-regular networks of arbitrary finite width | exact reconstruction | Helped |
| Identify first-layer critical hyperplanes, recover the first layer, then reduce the top two layers to depth-two reconstruction | Depth-three ReLU networks · δ-regular networks with d_1 ≤ d and non-zero top-layer partial derivatives | exact reconstruction | Helped |
| Allow arbitrary finite width instead of requiring linearly independent rows in the depth-two weights matrix | Depth-two ReLU extraction · General-position ReLU networks | network width supported | Helped |
| Add small random perturbations to network weights | Two- and three-layer ReLU networks · Networks with bounded weights | probability of δ-regularity: 1-2^{-d} | Helped |
Closest papers by meaning
- Understanding Deep Neural Networks with Rectified Linear Units (ICLR 2018 · similarity 0.89)
- Efficient Algorithms for Learning Depth-2 Neural Networks with General ReLU Activations (NeurIPS 2021 · similarity 0.88)
- Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity (NeurIPS 2019 · similarity 0.88)
- Expressiveness of Rectifier Networks (ICML 2016 · similarity 0.88)
- The inductive bias of ReLU networks on orthogonally separable data (ICLR 2021 · similarity 0.88)
- Most ReLU Networks Admit Identifiable Parameters (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.