Research map / Reinforcement learning / Policy gradient and actor-critic
Bayesian Bellman Operators
Matthew Fellows, Kristian Hartikainen, Shimon Whiteson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BBO posterior (RP-BBAC) instead of posterior over Q-functions with bootstrapping (BAC) (ablation) | randomised-prior Bayesian actor-critic · MountainCar-Continuous-v0 and cartpole-swingup_sparse | task success | Improved |
| use single Q-function (SAC*) instead of double-minimum-Q trick (SAC) | SAC · sparse-reward continuous control | seeds solving the task | Mixed |
| replace SAC* with RP-BBAC (deep exploration via BBO posterior) | SAC* · sparse-reward continuous control | seeds solving the task | Improved |
| use BBO gradient-based MSBBE minimisation instead of ignoring posterior dependence on omega | Bayesian policy evaluation · 20-Link Pendulum, Puddle World, Mountain Car | mean squared error stability | Improved |
| add prior regularisation | BBO policy evaluation · nonlinear policy evaluation tasks | policy evaluation performance | Improved |
| use BBO instead of local linearisation of nonlinear TDC/GTD2 | nonlinear policy evaluation · policy evaluation tasks | mean squared error | Improved |
| use nonlinear Gaussian BBO instead of TD(0) | Tsitsiklis counterexample · Tsitsiklis counterexample | convergence | Improved |
| use BBO instead of nonlinear TDC and GTD2 | Tsitsiklis counterexample · Tsitsiklis counterexample | convergence speed | Improved |
| vary randomised prior hyperparameters (ablation) | RP-BBAC · sparse-reward continuous control | task performance | No clear effect |
Closest papers by meaning
- Epistemic Bellman Operators (AAAI 2025 · similarity 0.91)
- Bayesian Exploration Networks (ICML 2024 · similarity 0.90)
- Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning (ICML 2025 · similarity 0.90)
- Inverting the Bellman Equation: From $Q$-Values to World Models (NeurIPS 2026 · similarity 0.88)
- Computationally Efficient RL under Linear Bellman Completeness for Deterministic Dynamics (ICLR 2025 · similarity 0.87)
- Parameterized Projected Bellman Operator (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
