Research map / Reinforcement learning / Policy gradient and actor-critic
Exponential Smoothing for Off-Policy Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed bound-optimized exponential-smoothing method instead of the compared baselines | Off-policy learning on MNIST, FashionMNIST, EMNIST, and CIFAR100 · Supervised-to-bandit image classification tasks | reward | Helped |
| Use regularization with α around 0.7 rather than α near 1 (ablation) | Exponential-smoothing OPL · Modest logging policies (η₀ between 0 and 0.5) | average reward | Helped |
| Use α near 1, approaching standard IPS, rather than stronger regularization (ablation) | Exponential-smoothing OPL · Good logging policies (η₀ between 0.5 and 1), with sufficiently large n | average reward | No clear effect |
| Use standard IPS (α = 1) rather than clipped IPS (ablation) | Off-policy learning · Relatively well-performing logging policies | reward | Helped |
| Set α adaptively from the bound rather than fixing it (ablation) | Exponential-smoothing OPL · Logging policies across η₀ values | reward | No clear effect |
| Use exponential smoothing with varying α instead of hard-clipping with varying τ (ablation) | Exponential-smoothing OPL versus the best baseline, Sakhi et al. 2 with Gaussian policies · Non-uniform logging policies (η₀ > 0) | reward | Helped |
| Set the hard-clipping threshold τ to 10^-5 (ablation) | Clipped-IPS baseline · Off-policy learning across logging policies | improvement over logging policy | Hurt |
| Use Gaussian policies instead of mixed-logit policies (ablation) | Compared OPL methods · Supervised-to-bandit image classification tasks | reward | Helped |
| Use mixed-logit rather than Gaussian policies (ablation) | Proposed OPL method · Supervised-to-bandit image classification tasks | reward | No clear effect |
| Use the baselines rather than the proposed method | Off-policy learning · Uniform logging policy (η₀ = 0) | reward | Hurt |
Closest papers by meaning
- PAC-Bayesian Offline Contextual Bandits With Guarantees (ICML 2023 · similarity 0.91)
- Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning (NeurIPS 2024 · similarity 0.89)
- Log-Sum-Exponential Estimator for Off-Policy Evaluation and Learning (ICML 2025 · similarity 0.88)
- Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness (ICML 2024 · similarity 0.87)
- Cross-Domain Off-Policy Evaluation and Learning for Contextual Bandits (ICLR 2025 · similarity 0.87)
- On the Optimal Sample Complexity of Offline Multi-Armed Bandits with KL Regularization (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.