Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Inference and Learning in Dynamic Decision Networks Using Knowledge Compilation

AAAI 2024 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Approximate inference in Bayesian neural networks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes mapl-cirup, which knowledge-compiles the Bellman update of dynamic decision networks into dynamic decision circuits evaluated with algebraic model counting, enabling exact value iteration and gradient-based reward learning.

Problem

Exact planning and learning in factored MDPs represented as dynamic decision networks is costly because repeated Bellman updates require recompiling symbolic representations, and intra-state dependencies cause blow-ups in existing solvers.

Approach

Compile the DDN transition structure once into a dynamic decision circuit and evaluate the Bellman update as an algebraic model counting call over a max-sum-product semiring inside value iteration, yielding differentiable circuits usable for gradient-based reward learning.

Key results

• mapl-cirup produces much smaller circuits than SPUDD on the benchmarks (e.g., 2932 nodes vs. 38163 for factory is not the same measure; circuit sizes reported per model in Table 1), and solves factory in 0.93s KC plus 0.105s VI versus SPUDD's 0.01s VI.
• Linear utility approximation reduces utility nodes from 2^|X| to |X| and enables larger chain/cross-stitch instances up to 9-10 variables.
• Reward learning on coffee example reduces average relative state error from 2.94 to 0.41.

Measured effects reported in the paper

Surprising findings

• SPUDD remains faster in value iteration time on the small benchmarks, while mapl-cirup's advantage is circuit compactness and differentiability; the utility function encoding is the main scalability bottleneck.

Limitations

• Utility function is represented explicitly with 2^|X| utility parameters, causing exponential blow-up and limiting scalability.
• Value iteration still iterates over all explicit states.
• Only reward parameters learned; probability parameter learning left as future integration work.

Experimental setup

Datasets: MDP instances from SPUDD repository (elevator, coffee, factory) monkey DDN example synthetic cross-stitch and chain DDN families sampled trajectories from coffee example

Benchmarks: monkey elevator coffee factory cross-stitch DDN family chain DDN family

Metrics: value iteration time total time including knowledge compilation circuit size (number of nodes) average relative parameter error average relative state error

Baselines: SPUDD SPUDD-isd

Builds on: algebraic model counting SPUDD Chavira and Darwiche encoding of Bayesian networks Derkinderen and De Raedt decision-making AMC Vlasselaer et al. dynamic Bayesian network knowledge compilation

Closest papers by meaning

• Martingale Posterior Neural Networks for Fast Sequential Decision Making (NeurIPS 2025 · similarity 0.86)
• Bayesian Exploration Networks (ICML 2024 · similarity 0.85)
• DeLLMa: Decision Making Under Uncertainty with Large Language Models (ICLR 2025 · similarity 0.84)
• Scalable Solutions for Decision-Making Systems Using Explainable Policy Representations (AAAI 2025 · similarity 0.84)
• Dynamic Knowledge Injection for AIXI Agents (AAAI 2024 · similarity 0.84)
• In Search of Trees: Decision-Tree Policy Synthesis for Black-Box Systems via Search (AAAI 2025 · similarity 0.84)