Research map / Learning theory and algorithms / Sample complexity
On the Optimal Memorization Power of ReLU Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed width-12 construction with weight-encoded data and bit extraction | Feedforward ReLU networks · Separated labeled samples | number of parameters: O\left(N^{2/3}\right) -> \tilde{O}\left(\sqrt{N}\right) | Helped |
| Bound the network depth by L and use subset subnetworks | ReLU memorization construction · Separated labeled samples, depth bounded by L | number of parameters: O\left( \frac{N}{L\sqrt{\log(L)}}\cdot \log(R) + d\right) | Helped |
| Limit weight bit complexity to B and concatenate subset subnetworks | ReLU memorization construction · Separated labeled samples, weight bit complexity bounded by B | number of parameters: \tilde{O}\left(\frac{N}{B}\right) | Helped |
| Use weights with bit complexity that grows with N | ReLU networks with N^{1-\epsilon} parameters · Shattering N points | minimum weight bit complexity: \Omega(N^{\epsilon}) | Helped |
Closest papers by meaning
- Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity (NeurIPS 2019 · similarity 0.94)
- OPTIMAL ROBUST MEMORIZATION WITH RELU NEURAL NETWORKS (ICLR 2024 · similarity 0.92)
- The Cost of Robustness: Tighter Bounds on Parameter Complexity for Robust Memorization in ReLU Nets (NeurIPS 2025 · similarity 0.91)
- Network size and size of the weights in memorization with two-layers neural networks (NeurIPS 2020 · similarity 0.91)
- On the Optimal Memorization Capacity of Transformers (ICLR 2025 · similarity 0.89)
- Generalizablity of Memorization Neural Network (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.