Research map / Optimization / Optimizers
Escaping the Gravitational Pull of Softmax
Jincheng Mei, Chenjun Xiao, Bo Dai, Lihong Li, Csaba Szepesvari, Dale Schuurmans
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace softmax with escort transform (EPG) for policy gradient on bandits | policy gradient · one-state MDPs, K=10/50/100 | convergence to optimal policy / suboptimality | Improved |
| replace softmax with escort transform (p=2) in policy gradient | policy gradient with ReLU network · Four-room with sub-goals | value / optimal goal reaching | Improved |
| replace softmax with escort transform (p=4) for misclassification-probability minimization | one-hidden-layer MLP · MNIST (policy gradient objective) | training objective and test error | Improved |
| replace softmax cross entropy with escort cross entropy (p=2) | one-hidden-layer MLP · MNIST | training objective speed and test error | Improved |
| use small escort exponent p instead of large p in EPG (ablation) | EPG · single-state MDP K=3, r=(0.2,0.9,1.0) | resistance to suboptimal corner attraction | Improved |
| use p=1 escort instead of p>=2 (ablation) | EPG · single-state MDP K=3 | smoothness of optimization curves | Hurt |
| mirror descent with KL regularization compared with EPG p=2 | EPG · single-state MDP K=3 | suboptimality trajectory | No clear effect |
Closest papers by meaning
- Beyond Softmax: A Natural Parameterization for Categorical Random Variables (ICML 2026 · similarity 0.82)
- In Defense of Softmax Parametrization for Calibrated and Consistent Learning to Defer (NeurIPS 2023 · similarity 0.82)
- Soft ascent-descent as a stable and flexible alternative to flooding (NeurIPS 2024 · similarity 0.81)
- Sigsoftmax: Reanalysis of the Softmax Bottleneck (NeurIPS 2018 · similarity 0.81)
- Adaptive Sampling for Efficient Softmax Approximation (NeurIPS 2024 · similarity 0.79)
- Rethinking Approximate Gaussian Inference in Classification (NeurIPS 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
