Research map / Label-efficient and robust learning / Recognition and long-tail
Depth Selection for Deep ReLU Nets in Feature Extraction and Generalization
Zhi Jun Han, Siquan Yu, Shao-Bo Lin, Ding‐Xuan Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase depth from 1 to 3 layers (ablation) | ReLU net · Square-feature regression (10-d) | MSE: 14837.713 -> 4999.949 | Improved |
| increase depth from 3 to 5 layers (ablation) | ReLU net · Square-feature regression (10-d) | MSE: 4999.949 -> 49579.938 | Hurt |
| increase depth from 3 to 11 layers (ablation) | ReLU net · Square-feature regression (10-d) | MSE: 4999.949 -> 3053474.62 | Hurt |
| increase depth from 1 to 4 layers (ablation) | ReLU net · Noisy radial regression (sigma^2=0.1) | MSE: 0.0923 -> 0.0694 | Improved |
| increase depth from 4 to 6 layers (ablation) | ReLU net · Noisy radial regression (sigma^2=0.1) | MSE: 0.0694 -> 0.1265 | Hurt |
| increase depth from 1 to 4 layers (ablation) | ReLU net · Synthetic seismic intensity | MSE: 0.0578 -> 0.0037 | Improved |
| increase depth from 4 to 6 layers (ablation) | ReLU net · Synthetic seismic intensity | MSE: 0.0037 -> 0.1684 | Hurt |
| increase depth from 1 to 5 layers (ablation) | deep net classifier · US earthquake intensity (real, 608 samples) | recognition rate (%): 57.41 -> 66.67 | Improved |
| increase depth from 5 to 7 layers (ablation) | deep net classifier · US earthquake intensity (real, 608 samples) | recognition rate (%): 66.67 -> 62.03 | Hurt |
| use 5-layer deep net instead of SVM | classifier · US earthquake intensity (real, 608 samples) | recognition rate (%): 62.96 -> 66.67 | Improved |
| use 3-layer net instead of shallow net on partially radial feature | ReLU net · Partially radial feature (k=2) | MSE: 1357.934 -> 777.817 | Improved |
| use 3-layer net instead of shallow net on partially radial feature | ReLU net · Partially radial feature (k=9) | MSE: 9154.373 -> 7098.902 | Improved |
Closest papers by meaning
- Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks (ICLR 2024 · similarity 0.84)
- Is Deeper Better only when Shallow is Good? (NeurIPS 2019 · similarity 0.83)
- Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss (ICML 2024 · similarity 0.83)
- Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification (NeurIPS 2025 · similarity 0.83)
- Feature Contamination: Neural Networks Learn Uncorrelated Features and Fail to Generalize (ICML 2024 · similarity 0.82)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
