Research map / Reinforcement learning / MDPs and planning

Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach

AAAI 2025 · Reinforcement learning › MDPs and planning · approach: POMDP policy optimization · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Learns interpretable ASP belief-based policy specifications with ILASP from POMCP execution traces and uses them as soft heuristics in POMCP, DESPOT, AdaOPS and POMCPOW on rocksample and pocman.

Problem

Online POMDP solvers (MCTS-based) degrade in domains with large action spaces and long planning horizons unless task-specific policy heuristics are hand-designed; learning such heuristics with neural networks needs much data and yields uninterpretable policies.

Approach

Lift belief-action pairs from POMCP traces to ASP atoms via a feature map, learn normal rules and weak-constraint preferences with ILASP, and use the resulting logical specifications as probabilistic action guidance in UCT, rollout, and DESPOT lower bounds.

Key results

• Learned ASP heuristics yield performance superior to neural networks and similar to optimal handcrafted heuristics with lower computational time on rocksample and pocman, per the abstract.
• Learned specifications generalize to larger rocksample grids and more rocks, and to larger pocman maps with more and more aggressive ghosts, not seen in training.
• Training uses 1000 POMCP traces generated in about 12 minutes, with ILASP learning requiring few minutes.

Surprising findings

• Learned heuristics are robust to bad examples because soft probabilistic guidance preserves exploration of non-entailed actions.

Limitations

• Relies on user-defined domain-level features; incomplete feature sets degrade heuristic quality.
• Assumes actions' axioms are independent of other actions in the rule bodies.
• Pocman not supported in AdaOPS; cross-planner comparison limited due to different implementations (C++ vs Julia).

Experimental setup

Datasets: POMCP-generated rocksample traces (N=12, M=4) POMCP-generated pocman traces (10x10, G=2)

Benchmarks: rocksample pocman

Metrics: discounted return computational time planning horizon

Baselines: POMCP without heuristics DESPOT AdaOPS POMCPOW handcrafted POMCP heuristics (Silver et al. 2010) AIS deep RL (Subramanian et al. 2022) neural network policies

Builds on: POMCP DESPOT ILASP mazzi2023learning preliminary work AdaOPS POMCPOW

Closest papers by meaning

• A Surprisingly Simple Continuous-Action POMDP Solver: Lazy Cross-Entropy Search Over Policy Trees (AAAI 2024 · similarity 0.88)
• Sequential Monte Carlo for Policy Optimization in Continuous POMDPs (NeurIPS 2025 · similarity 0.88)
• Partially Observable Hierarchical Reinforcement Learning with AI Planning (Student Abstract) (AAAI 2024 · similarity 0.87)
• Scalable Policy-Based RL Algorithms for POMDPs (NeurIPS 2025 · similarity 0.87)
• Simplifying Complex Observation Models in Continuous POMDP Planning with Probabilistic Guarantees and Practice (AAAI 2024 · similarity 0.87)
• Revelations: A Decidable Class of POMDPs with Omega-Regular Objectives (AAAI 2025 · similarity 0.87)