Research map / Optimization / Optimizers
Diving into the shallows: a computational perspective on large-scale shallow learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Precondition the kernel iteration using approximate top eigenvectors | Pegasos / SGD kernel learning · HINT-S, TIMIT, MNIST-8M, and HINT-M | c-error or mse after 10 epochs: 11.7% -> 10.0%; 33.0% -> 31.7%; 1.1% -> 0.8%; 2.7e-2 -> 2.3e-2 | Helped |
| Precondition the random Fourier feature iteration using approximate top eigenvectors | SGD-RF · HINT-S, TIMIT, MNIST-8M, and HINT-M | c-error or mse after 10 epochs: 11.5% -> 10.3%; 33.3% -> 32.6%; 1.0% -> 0.8%; 2.7e-2 -> 2.4e-2 | Helped |
| Use EigenPro rather than Pegasos with a Gaussian kernel | Kernel classification · MNIST | epochs to reach KRLS classification error: 77 -> 7 | Helped |
| Use a smaller Gaussian-kernel bandwidth (σ² = 5 instead of σ² = 25) (ablation) | EigenPro on MNIST · MNIST | test c-error: 1.20% -> 1.83% | Hurt |
| Use a smaller Gaussian-kernel bandwidth (σ² = 5 instead of σ² = 25) (ablation) | EigenPro on MNIST · MNIST | epochs to reach best test c-error: 20 -> 5 | Helped |
| Use Laplace rather than Gaussian kernel (ablation) | KRLS · MNIST | best classification error: 1.3% -> 1.6% | Hurt |
| Use Cauchy rather than Gaussian kernel (ablation) | KRLS · CIFAR-10 | best classification error: 49.5% -> 48.4% | Helped |
| Train with more epochs (ablation) | Pegasos with Gaussian kernel · MNIST | test c-error: 5.21% -> 1.24% | Helped |
| Train with more epochs (ablation) | EigenPro with Gaussian kernel · MNIST | test c-error: 2.03% -> 1.20% -> 1.23% | Mixed |
Closest papers by meaning
- Learning Gradient Descent: Better Generalization and Longer Horizons (ICML 2017 · similarity 0.90)
- Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries (ICLR 2025 · similarity 0.90)
- Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks (ICLR 2018 · similarity 0.89)
- Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization (ICLR 2020 · similarity 0.89)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.89)
- Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.