Research map / Learning theory and algorithms / Kernels and regression theory
A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the joint large-n,p,N deterministic-equivalent analysis instead of the large-N Gaussian-kernel prediction | RFF ridge regression · MNIST (class 3 versus 7) | training MSE prediction fit | Helped |
| Increase feature dimension N through the interpolation threshold with very small regularization | RFF ridge regression · MNIST (class 3 versus 7) | test MSE | Mixed |
| Use larger positive ridge regularization | RFF ridge regression · MNIST (class 3 versus 7) | test MSE | Helped |
| Train in the over-parameterized regime, 2N > n, rather than the under-parameterized regime | RFF ridge regression · MNIST (class 3 versus 7) | test MSE | Helped |
| Choose a positive regularization parameter instead of using the ridgeless limit | RFF ridge regression · MNIST (class 3 versus 7), near 2N = n | test MSE | Helped |
| Make test inputs sufficiently different from training inputs rather than identical | RFF ridge regression · Test data near 2N = n, ridgeless limit | test MSE | Hurt |
| Add Gaussian white noise to training inputs to form test inputs, with noise variance above lambda | RFF ridge regression · Perturbed training data | test MSE | Hurt |
| Apply the analysis to Fashion-MNIST and Kannada-MNIST | RFF ridge regression · Fashion-MNIST and Kannada-MNIST | test MSE prediction fit | Helped |
| Increase N through 2N=n with very small regularization | RFF ridge regression · Fashion-MNIST and Kannada-MNIST | test MSE | Mixed |
Closest papers by meaning
- Towards a Unified Analysis of Random Fourier Features (ICML 2019 · similarity 0.91)
- On The Relative Error of Random Fourier Features for Preserving Kernel Distance (ICLR 2023 · similarity 0.91)
- Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees (ICML 2017 · similarity 0.90)
- TRF: Learning Kernels with Tuned Random Features (AAAI 2022 · similarity 0.90)
- Sharp Analysis of Random Fourier Features in Classification (AAAI 2022 · similarity 0.90)
- Efficiently Learning Significant Fourier Feature Pairs for Statistical Independence Testing (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.