Research map / Reinforcement learning / Policy gradient and actor-critic
Bayesian Risk-Averse Q-Learning with Streaming Observations
Yuhao Wang, Enlu Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bayesian-risk (VaR/CVaR) Q-learning with posterior updated from streaming real observations instead of distributionally robust Q-learning with a fixed ambiguity set | BRQL-VaR and BRQL-CVaR · Coin Toss and Inventory Management (streaming observations) | value function in real environment (expected performance and variation) | Improved |
| Use risk functional VaR/CVaR rather than the risk-neutral expectation over the posterior (ablation) | BRQL · Coin Toss and Inventory Management (streaming observations) | expected value function | Hurt |
| Use risk functional VaR/CVaR rather than the risk-neutral expectation over the posterior (ablation) | BRQL · Coin Toss and Inventory Management (streaming observations) | variation of value function (95% CI width) | Improved |
| Use risk functional VaR/CVaR rather than risk-neutral mean when test demand is more adversarial (Poisson parameter below 3) (ablation) | BRQL (insufficient data, fixed BRMDP) · Inventory Management with differing Poisson demand | value function | Improved |
| Use VaR/CVaR risk functional instead of worst-case ambiguity set (DRQL) in the adversarial regime | BRQL vs DRQL-KL and DRQL-Wass · Inventory Management with demand Poisson parameter below 3 | value function | Hurt |
| Use VaR/CVaR risk functional instead of worst-case ambiguity set (DRQL) in the non-adversarial regime | BRQL vs DRQL-KL and DRQL-Wass · Inventory Management with demand Poisson parameter near or above 3 | value function | Improved |
| Vary risk level alpha between 0.2 and 0.4 (ablation) | BRQL-VaR and BRQL-CVaR · Coin Toss | value function | No clear effect |
Closest papers by meaning
- Distributionally Robust $Q$-Learning (ICML 2022 · similarity 0.89)
- Risk-Averse Bayes-Adaptive Reinforcement Learning (NeurIPS 2021 · similarity 0.89)
- Risk-Averse Total-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Bayesian Exploration Networks (ICML 2024 · similarity 0.86)
- Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis (AAAI 2024 · similarity 0.86)
- Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
