Research map / Optimization / Implicit bias and generalization
Student Specialization in Deep Rectified Networks With Finite Width and Input Dimension
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase student width relative to teacher width (ablation) | ReLU student · Synthetic teacher-student network | teacher-node recovery rate | Helped |
| use only 1× student nodes instead of over-realization (ablation) | ReLU student · Synthetic teacher-student network | teacher-node recovery rate | Hurt |
| increase the number of training samples (ablation) | Student network · Synthetic teacher-student network | node correlation | Helped |
| use teacher-aware rather than teacher-agnostic data augmentation (ablation) | Student network · Synthetic teacher-student network | convergence | Helped |
| use teacher-aware rather than teacher-agnostic data augmentation (ablation) | Student network · Synthetic teacher-student network | generalization | Helped |
| increase student over-realization (ablation) | Student network · CIFAR-10 evaluation set | generalization | Helped |
| train longer (ablation) | Over-realized student · Synthetic two-layer teacher-student network | teacher-node recovery rate | Helped |
| increase teacher-node strength (ablation) | Student specialization · Synthetic two-layer teacher-student network | teacher-node recovery rate | Helped |
| increase teacher polarity, making teacher-node strengths more unequal (ablation) | Student specialization · Synthetic two-layer teacher-student network | teacher-node recovery rate | Mixed |
Closest papers by meaning
- On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Setting (ICML 2021 · similarity 0.91)
- Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methods (ICLR 2023 · similarity 0.91)
- How a Student becomes a Teacher: learning and forgetting through Spectral methods (NeurIPS 2023 · similarity 0.89)
- On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths (ICML 2021 · similarity 0.88)
- Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training (ICML 2025 · similarity 0.87)
- On the Hardness of Training Deep Neural Networks Discretely (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.