Research map / Learning theory and algorithms / Kernels and regression theory
Adaptive Sampled Softmax with Kernel Based Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use quadratic-kernel sampling instead of uniform sampling | Sampled softmax · Penn Tree Bank, YouTube10k, YouTube100k | sample efficiency for matching full-softmax quality: one to two orders of magnitude less samples | Helped |
| Increase the sample size for uniform or quadratic sampling | Sampled softmax · Penn Tree Bank, YouTube10k, YouTube100k | final model quality | Helped |
| Use softmax sampling instead of uniform or quadratic sampling | Sampled softmax · Penn Tree Bank, YouTube10k, YouTube100k | bias relative to full softmax | Helped |
| Use quadratic sampling instead of uniform sampling | Sampled softmax · Penn Tree Bank, YouTube10k, YouTube100k | final loss | Helped |
| Increase sample size after enough samples remove most bias | Sampled softmax · Penn Tree Bank, YouTube10k, YouTube100k | convergence speed | No clear effect |
| Use uniform sampling rather than quadratic or softmax sampling | Sampled softmax · Penn Tree Bank, YouTube10k, YouTube100k | final loss | Hurt |
| Use quadratic sampling rather than softmax sampling | Sampled softmax · Penn Tree Bank, YouTube10k, YouTube100k | final loss | Hurt |
| Replace standard softmax with absolute softmax | Full softmax · Penn Tree Bank, YouTube10k, YouTube100k | model quality | No clear effect |
| Increase the number of training epochs while keeping a biased sampling distribution and fixed sample size | Sampled softmax · Penn Tree Bank, YouTube10k, YouTube100k | final loss | No clear effect |
Closest papers by meaning
- Sampled Estimators For Softmax Must Be Biased (NeurIPS 2025 · similarity 0.90)
- Adaptive Sampling for Efficient Softmax Approximation (NeurIPS 2024 · similarity 0.88)
- Sparse Attention as Compact Kernel Regression (NeurIPS 2026 · similarity 0.85)
- Kernel Learning for Sample Constrained Black-Box Optimization (AAAI 2025 · similarity 0.85)
- MultiMax: Sparse and Multi-Modal Attention Learning (ICML 2024 · similarity 0.84)
- Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.