Research map / Learning theory and algorithms / Kernels and regression theory
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a fully connected NTK SVM instead of Random Forest | UCI 90 datasets · UCI 90 datasets | Average Accuracy: 81.56% -> 81.95% | Helped |
| Use CNTK instead of ResNet-34 with 320 training samples | CIFAR-10 · CIFAR-10 (320 training samples) | accuracy: 33.15\% -> 36.57\% | Helped |
| Use CNTK instead of ResNet-34 with 1280 training samples | CIFAR-10 · CIFAR-10 (1280 training samples) | accuracy: 49.14\% -> 48.86\% | Hurt |
| Increase CNTK depth from 5 to 14 convolutional layers (ablation) | CNTK on CIFAR-10 · CIFAR-10 (320 training samples) | accuracy: 35.26\% -> 36.57\% | Helped |
| Replace the linear SVM with CNTK | VOC07 few-shot classification using conv5 features · VOC07 test, conv5 features, k=1 | mAP: $49.46 \pm 4.71$ -> $50.04 \pm 4.28$ | Helped |
| Use CNTK instead of linear SVM on intermediate conv3 features | VOC07 few-shot classification · VOC07 test, conv3 features, k=1 | mAP: $14.38 \pm 1.98$ -> $15.42 \pm 1.90$ | Helped |
| Use deeper CNTK rather than one convolutional layer on VOC07 conv5 features (ablation) | VOC07 few-shot classification · VOC07 test, conv5 features, k=1 | mAP: $50.04 \pm 4.28$ -> $48.00 \pm 4.41$ | Hurt |
| Use deeper CNTK rather than one convolutional layer on VOC07 conv3 features with k=7 (ablation) | VOC07 few-shot classification · VOC07 test, conv3 features, k=7 | mAP: The table reports 1-layer $27.98 \pm 2.46$ and 3-layer $28.24 \pm 2.22$. | Helped |
| Use NTK initialization rather than He initialization | Fully connected neural network on UCI · UCI 90 datasets | Average Accuracy: 80.88\% -> 81.02\% | Helped |
Closest papers by meaning
- Scaling Neural Tangent Kernels via Sketching and Random Features (NeurIPS 2021 · similarity 0.92)
- Fast Finite Width Neural Tangent Kernel (ICML 2022 · similarity 0.91)
- On Exact Computation with an Infinitely Wide Neural Net (NeurIPS 2019 · similarity 0.91)
- The Surprising Effectiveness of Infinite-Width NTKs for Characterizing and Improving Model Training (AAAI 2025 · similarity 0.90)
- Optimized Random Features for the Neural Tangent Kernel (Student Abstract) (AAAI 2025 · similarity 0.88)
- On the Impacts of the Random Initialization in the Neural Tangent Kernel Theory (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.