Research map / Architectures and efficiency / RNNs and normalization
Fast Neural Kernel Embeddings for General Activations
Insu Han, Amir Zandieh, Jaehoon Lee, Roman Novak, Lechao Xiao, Amin Karbasi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use scaled normalized Gaussian instead of ReLU for CNTK features | Myrtle5 CNTK on CIFAR-10 · CIFAR-10 | test accuracy: 75.56% -> 78.13% | Improved |
| Approximate CNTK with sketching rather than compute exact CNTK | Myrtle5 CNTK on CIFAR-10 · CIFAR-10, 60,000 training examples | runtime: 151 GPU hours -> 1.4 GPU hours; 106× speedup | Improved |
| Use CNTK approximation with 20,000 training examples instead of exact computation | Myrtle5 CNTK on CIFAR-10 · CIFAR-10, 20,000 training examples | runtime: 12× faster without loss of accuracy | Improved |
| Approximate dual kernels with truncated Hermite expansion rather than Monte Carlo estimation | Dual kernels for ReLU, Abs, sin, Gaussian, erf, and GeLU · Synthetic dataset, n=1,000, d=256 | relative Frobenius error | Improved |
Closest papers by meaning
- Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK (JMLR 2024 · similarity 0.89)
- Finite Versus Infinite Neural Networks: an Empirical Study (NeurIPS 2020 · similarity 0.89)
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks (NeurIPS 2025 · similarity 0.88)
- Deep Equals Shallow for ReLU Networks in Kernel Regimes (ICLR 2021 · similarity 0.88)
- Convolutional Deep Kernel Machines (ICLR 2024 · similarity 0.87)
- ReLUs Are Sufficient for Learning Implicit Neural Representations (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
