Research map / Multi-agent systems and games / Human-AI and goal-conditioned
HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces HASARD, a ViZDoom-based benchmark of six vision-based safe RL environments with three difficulty levels, and evaluates six PPO-based safe RL baselines on it.
Problem
Existing safe RL benchmarks are either 2D toy problems, proprioceptive robotics control, or simple 3D navigation, with few supporting embodied egocentric vision-based safe RL in complex 3D settings.
Approach
Builds lightweight, stochastic ViZDoom environments with reward and cost functions tied to safety-relevant behaviors (e.g., fall damage, collateral harm, lava), with multiple difficulty levels and simplified discrete action spaces, integrated with Sample-Factory for fast training.
Key results
- PPOPID consistently satisfies cost thresholds and most often outperforms other baselines, making it arguably the most effective method on HASARD.
- PPOLag closely adheres to safety thresholds but often yields the lowest rewards.
- Incremental curriculum across difficulty levels gives nearly threefold reward increase in Remedy Rush (18.82 vs 6.65) versus regular Level 3 training at equal timesteps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PPOPID rather than unconstrained PPO | HASARD baseline agents · HASARD Level 1 tasks | cost | Helped |
| Train sequentially across difficulty levels rather than directly on Level 3 | PPOPID · Remedy Rush Level 3 | reward: 6.65 -> 18.82 | Helped |
| Train sequentially across difficulty levels rather than directly on Level 3 | PPOPID · Collateral Damage Level 3 | reward: 11.46 -> 15.32 | Helped |
| Use segmented observations instead of raw RGB observations | PPOPID · Armament Burden Level 2 | reward: +21% | Helped |
| Concatenate depth information to the RGB observation | PPO · Precipice Plunge Level 2 | reward: +34% | Helped |
| Tighten the safety cost limit | PPOLag · HASARD Level 1 tasks | reward | Hurt |
Surprising findings
- PPOCost sometimes achieves rewards comparable to or above constrained methods but without guaranteed cost satisfaction, and PPOSauté and P3O exceed cost thresholds in several environments.
- Simply adding depth buffer channel increased PPO reward by 34% in Precipice Plunge Level 2, suggesting depth perception is hard to learn from RGB alone.
Limitations
- ViZDoom does not capture real-world physics and visual realism, limiting direct transfer to real applications.
- Most actions are discrete, simplifying fine control challenges.
- Environments have only a single objective and single safety constraint.
Experimental setup
Benchmarks: HASARD Armament Burden HASARD Volcanic Venture HASARD Remedy Rush HASARD Collateral Damage HASARD Precipice Plunge HASARD Detonator's Dilemma
Metrics: episode return (reward) episode cost
Baselines: PPO PPOCost PPOLag PPOSauté PPOPID P3O
Builds on: PPO PPO-Lagrangian Sauté RL P3O Sample-Factory ViZDoom
Closest papers by meaning
- AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions (CVPR 2026 · similarity 0.88)
- DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization (ICLR 2024 · similarity 0.87)
- VIKI‑R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors (NeurIPS 2024 · similarity 0.86)
- D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigation (CVPR 2026 · similarity 0.86)
- METRA: Scalable Unsupervised RL with Metric-Aware Abstraction (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.