Research map / Reinforcement learning / Visual and representation RL
Neurosymbolic Reinforcement Learning: Playing MiniHack with Probabilistic Logic Shields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Demo paper presenting a codebase and web platform for designing custom MiniHack environments where PPO agents use ProbLog-based probabilistic logic shields for safe RL, compared against reward shaping.
Problem
Safe reinforcement learning in uncertain environments requires enforcing logical safety constraints, but practical implementations of probabilistic logic shields are limited and require specialised knowledge.
Approach
Integrates probabilistic logic shields (ProbLog rules plus sensor functions that renormalise a neural policy) into MiniHack Gym environments via Stable Baselines3 PPO, packaged as an extensible codebase with a Jupyter notebook and an interactive web demo.
Key results
- Shielded PPO agents converge faster in the Cliff environment than reward-shaped agents, since the shield also guides exploration.
- Shielded agents remain safe and generalize to test-time changes (extra monsters or lava) without retraining, as reported qualitatively in the demo.
- Demo paper with qualitative comparisons only; no quantitative numbers are reported in the text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use probabilistic logic shielding instead of reward shaping for safety | MiniHack agents · Cliff | training convergence | Helped |
| use probabilistic logic shielding instead of reward shaping for safety | MiniHack agents · MiniHack environments | generalization | Helped |
| use probabilistic logic shielding instead of reward shaping for safety | MiniHack agents · MiniHack environments | safety | Helped |
| use a shield with perfect sensors and rules that prioritize collecting the key before opening the door | MiniHack agent · Key Room | agent exhaustion | Helped |
| use probabilistic logic shielding to avoid monsters | MiniHack agent · Monster Room | goal-reaching while avoiding monsters | Helped |
| use probabilistic logic shielding with stochastic transitions | MiniHack agent · Lava Lake | safety | Helped |
| add hazards or change the shield at test time without retraining | shielded MiniHack agent · MiniHack environments | safety | Helped |
Limitations
- Assumes perfect sensors in the experiments (MiniHack provides symbolic observations).
- Evaluation is qualitative and demonstrated on a few custom environments.
- Targeted at expert users; ease of use for non-experts is claimed but not measured.
Experimental setup
Datasets: MiniHack custom environments (Cliff, Key Room, Monster Room, Lava Lake)
Benchmarks: Cliff Key Room Monster Room Lava Lake
Metrics: training convergence safety generalization
Baselines: reward shaping PPO agent
Builds on: probabilistic logic shields MiniHack Stable Baselines3 ProbLog PPO
Closest papers by meaning
- Probabilistic Shielding for Safe Reinforcement Learning (AAAI 2025 · similarity 0.86)
- PUZZLES: A Benchmark for Neural Algorithmic Reasoning (NeurIPS 2024 · similarity 0.85)
- On the Hardness of Probabilistic Neurosymbolic Learning (ICML 2024 · similarity 0.85)
- BlendRL: A Framework for Merging Symbolic and Neural Policy Learning (ICLR 2025 · similarity 0.85)
- PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning (NeurIPS 2024 · similarity 0.85)
- Relational Neurosymbolic Markov Models (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.