Research map / Learning theory and algorithms / Kernels and regression theory
Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networks
Yuval Belfer, Amnon Geifman, Meirav Galun, Ronen Basri
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase depth from 5 to 100 layers (ablation) | FC-NTK · SVHN | accuracy: 74.44 -> 21.12 | Hurt |
| increase depth from 5 to 100 layers (ablation) | FC-NTK · CIFAR-10 | accuracy: 58.28 -> 48.27 | Hurt |
| increase depth from 5 to 100 layers (ablation) | FC-NTK · UCI (90 datasets) | accuracy: 85.54 -> 80.87 | Hurt |
| use decaying alpha=1/L instead of constant alpha=1 in ResNTK (ablation) | ResNTK, 100 layers · UCI (90 datasets) | accuracy: 79.91 -> 85.38 | Improved |
| use ResNTK alpha=1/L instead of FC-NTK at 100 layers | ResNTK · CIFAR-10 | accuracy: 48.27 -> 57.53 | Improved |
| use ResNTK alpha=1/sqrt(L) instead of 1/L (ablation) | ResNTK, 5 layers · SVHN | accuracy: 73.62 -> 78.36 | Improved |
| use ResNTK instead of FC-NTK at shallow depth (5 layers) | ResNTK alpha=1/L · UCI (90 datasets) | accuracy: 85.54 -> 85.59 | No clear effect |
Closest papers by meaning
- A Kernel Perspective of Skip Connections in Convolutional Networks (ICLR 2023 · similarity 0.90)
- On the Similarity between the Laplace and Neural Tangent Kernels (NeurIPS 2020 · similarity 0.90)
- Characterizing the spectrum of the NTK via a power series expansion (ICLR 2023 · similarity 0.89)
- Deep Equals Shallow for ReLU Networks in Kernel Regimes (ICLR 2021 · similarity 0.88)
- Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks (NeurIPS 2020 · similarity 0.88)
- Eigen Analysis of Conjugate Kernel and Neural Tangent Kernel (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
