Research map / Architectures and efficiency / RNNs and normalization
Rank Diminishing in Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Compose additional almost-everywhere-smooth layers | Deep neural networks · General networks | rank | Hurt |
| Use layer components with rank-deficient structure, such as normalization or linear layers | Deep neural networks · Network components | rank | Hurt |
| Couple many functions drawn independently from a fixed distribution | Deep neural networks · Equal-dimensional function compositions | numerical rank | Hurt |
| Project final features onto top principal components | ImageNet classifiers · ImageNet | classification accuracy | No clear effect |
Closest papers by meaning
- Neural collapse vs. low-rank bias: Is deep neural collapse really optimal? (NeurIPS 2024 · similarity 0.87)
- Low Rank Gradients and Where to Find Them (NeurIPS 2025 · similarity 0.86)
- The Persistence of Neural Collapse Despite Low-Rank Bias (NeurIPS 2025 · similarity 0.86)
- Generalization Bounds for Rank-sparse Neural Networks (NeurIPS 2025 · similarity 0.85)
- Implicit bias of SGD in $L_2$-regularized linear DNNs: One-way jumps from high to low rank (ICLR 2024 · similarity 0.85)
- Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.