Research map / Optimization / Optimizers
A Baseline for Any Order Gradient Estimation in Stochastic Computation Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add any-order baseline B (second-order term B(2) on top of first-order baseline) (ablation) | DiCE multi-agent second-order gradient estimator · Iterated prisoner's dilemma | correlation with exact Hessian (1000 samples): 0.29 -> 0.97 | Helped |
| add second order baseline term B(2) to first order baseline (ablation) | Higher order (3rd and 4th) gradient estimators · Iterated prisoner's dilemma | mean correlation with true gradients (1000 samples) | Helped |
| use first order baseline only (no new term) for higher order gradients (ablation) | Fourth order gradient estimator · Iterated prisoner's dilemma | correlation with true gradient | No clear effect |
| add any-order baseline (reduces required sample size for 0.97 Hessian correlation) | DiCE second-order Hessian estimation · Iterated prisoner's dilemma | samples required for 0.97 correlation: 100k -> 1k | Helped |
| add second order baseline B(2) (ablation) | LOLA-DiCE · IPD LOLA-DiCE | average per-step return: without baseline ~-1.6 (random ~-1.5); with baseline agents learn | Helped |
Closest papers by meaning
- DiCE: The Infinitely Differentiable Monte Carlo Estimator (ICML 2018 · similarity 0.95)
- Loaded DiCE: Trading off Bias and Variance in Any-Order Score Function Gradient Estimators for Reinforcement Learning (NeurIPS 2019 · similarity 0.92)
- Total stochastic gradient algorithms and applications in reinforcement learning (NeurIPS 2018 · similarity 0.91)
- ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update (ICLR 2024 · similarity 0.90)
- Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Relaxed Stationary Distribution Correction Estimation for Improved Offline Policy Optimization (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.