Research map / Optimization / Implicit bias and generalization
Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NGD instead of GD for NCE (ablation) | NCE · 1d Gaussian mean estimation | parameter distance | Helped |
| Use NGD instead of GD for NCE (ablation) | NCE · 16d Gaussian mean estimation | parameter distance | Helped |
| Replace NCE log loss with the eNCE exponential loss (ablation) | NGD · 1d Gaussian mean estimation | parameter distance | Helped |
| Replace NCE log loss with the eNCE exponential loss (ablation) | NGD · 16d Gaussian mean estimation | parameter distance | Helped |
| Use NGD instead of GD for NCE (ablation) | NCE · MNIST | loss | Helped |
| Use NGD instead of GD for eNCE (ablation) | eNCE · MNIST | loss | Helped |
| Use standard GD with a far-away noise distribution instead of normalized updates (ablation) | NCE on 1d Gaussian mean estimation · 1d Gaussian mean estimation | convergence steps | Hurt |
| Use Newton's method with the standard globally safe step size (ablation) | NCE on 1d Gaussian mean estimation · 1d Gaussian mean estimation | convergence steps | Hurt |
| Replace the NCE log loss with the eNCE exponential loss (ablation) | NCE objective · Exponential-family distributions | Hessian condition number | Helped |
| Use NGD with the eNCE objective rather than standard GD (ablation) | eNCE on exponential-family distributions · Exponential-family distributions | convergence rate | Helped |
Closest papers by meaning
- A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation (ICML 2025 · similarity 0.91)
- Is Noise Conditioning Necessary for Denoising Generative Models? (ICML 2025 · similarity 0.86)
- Near-Optimality of Contrastive Divergence Algorithms (NeurIPS 2024 · similarity 0.86)
- Test-Time Scaling of Diffusion Models via Noise Trajectory Search (NeurIPS 2025 · similarity 0.86)
- Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Training (NeurIPS 2024 · similarity 0.85)
- Your contrastive learning problem is secretly a distribution alignment problem (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.