Research map / Architectures and efficiency / CNN design
The Hessian perspective into the Nature of Convolutional Neural Networks
Sidak Pal Singh, Thomas Hofmann, Bernhard Schölkopf
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use convolutional layers with local connectivity and weight sharing | CNN Hessian rank · Deep CNNs | Hessian rank growth relative to parameter count | Improved |
| Add weight sharing to a locally connected network (ablation) | One-hidden-layer linear network with non-overlapping patches · Hessian rank bounds | Hessian rank | Improved |
| Use local connectivity rather than full connectivity (ablation) | One-hidden-layer network with non-overlapping patches · Hessian rank bounds | Hessian rank | Mixed |
| Increase the number of channels | CNN · Hessian rank versus channel count | Hessian rank | Mixed |
| Vary filter size (ablation) | One-hidden-layer CNN · Hessian rank bounds | Hessian rank | Mixed |
| Use ReLU rather than linear activations (ablation) | CNN · Empirical Hessian rank upper-bound verification | Hessian rank bound validity | No clear effect |
| Use cross-entropy rather than MSE loss (ablation) | CNN · Empirical Hessian rank upper-bound verification | Hessian rank bound validity | No clear effect |
| Use larger networks with more channels (ablation) | CNN · Hessian rank bound tightness across filter sizes | Hessian rank bound tightness | Improved |
Closest papers by meaning
- Analytic Insights into Structure and Rank of Neural Network Hessian Maps (NeurIPS 2021 · similarity 0.92)
- Investigating the Overlooked Hessian Structure: From CNNs to LLMs (ICML 2025 · similarity 0.91)
- Linear CNNs Discover the Statistical Structure of the Dataset Using Only the Most Dominant Frequencies (ICML 2023 · similarity 0.88)
- What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis (ICLR 2025 · similarity 0.87)
- Loss Landscape Characterization of Neural Networks without Over-Parametrization (NeurIPS 2024 · similarity 0.85)
- Position: A Theory of Deep Learning Must Include Compositional Sparsity (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
