Research map / Reinforcement learning / Policy gradient and actor-critic
A Risk-Sensitive Approach to Policy Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use pessimistic Wang weighting (η=0.5) instead of the risk-neutral expected-reward objective (ablation) | Risk-sensitive policy optimization · PointGoal2, CarButton2, PointPush2 | Expected Reward: 15.6 ± 0.3 -> 17.9 ± 0.4; 11.5 ± 1.2 -> 16.7 ± 0.5; 4.1 ± 0.6 -> 9.3 ± 2.1 | Helped |
| Use pessimistic Wang weighting (η=0.5) instead of risk-neutral training (ablation) | Risk-sensitive policy optimization · PointGoal2, CarButton2, PointPush2 | CPT Value: 3.7 ± 0.3 -> 5.7 ± 0.5; 0.5 ± 1.5 -> 5.2 ± 0.8; 1.0 ± 0.5 -> 4.5 ± 1.3 | Helped |
| Use pessimistic rather than optimistic weighting (ablation) | Risk-sensitive policy optimization · PointGoal2, CarButton2, PointPush2 | Expected Reward: 11.0 ± 3.2 -> 17.9 ± 0.4; 8.7 ± 1.0 -> 16.7 ± 0.5; 3.2 ± 2.1 -> 9.3 ± 2.1 | Helped |
| Add utility-to-go and a neural-network baseline to the base risk-sensitive gradient (ablation) | CPT value optimization · Safety Gym variance-reduction experiments | Sample efficiency | Helped |
| Add generalized advantage estimation after utility-to-go and a neural-network baseline (ablation) | CPT value optimization · Safety Gym variance-reduction experiments | Sample efficiency | Helped |
| Add trust-region clipping after generalized advantage estimation (ablation) | CPT value optimization · Safety Gym variance-reduction experiments | Sample efficiency | Helped |
| Use the paper's clipping form rather than PPO's clipping form (ablation) | Risk-sensitive policy optimization · Figures 3 and 4 environments | Performance | Helped |
| Increase pessimism (η) in the risk profile (ablation) | Risk-sensitive policy optimization · PointButton2 and trends across Safety Gym environments | Accumulated reward | Mixed |
| Increase pessimism (η) in the risk profile (ablation) | Risk-sensitive policy optimization · PointButton2 and constrained Safety Gym environments | Cost | Helped |
Closest papers by meaning
- Risk-Conditioned Reinforcement Learning: A Generalized Approach for Adapting to Varying Risk Measures (AAAI 2024 · similarity 0.91)
- Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning (ICML 2024 · similarity 0.91)
- Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learning (ICML 2025 · similarity 0.90)
- A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalents (ICML 2025 · similarity 0.89)
- Reward Penalties on Augmented States for Solving Richly Constrained RL Effectively (AAAI 2024 · similarity 0.89)
- Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.