Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Martingale Posterior Neural Networks for Fast Sequential Decision Making
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes predictive-first online neural network learning via martingale posteriors, using linearized Kalman-style low-rank covariance updates (HiLoFi, LoLoFi, LRKF) and sampling directly from the one-step posterior predictive for bandit and Bayesian optimization decisions.
Problem
Efficient uncertainty-aware online learning of neural networks for sequential decision making, where Bayesian neural networks with parameter-space posteriors are slow and often misspecified.
Approach
Work directly with a neural-network-parameterized one-step-ahead posterior predictive updated by single-pass frequentist EKF-style low-rank recursions (full-rank last layer, low-rank hidden layers), and sample outcomes from this predictive for decisions instead of sampling parameters as in Thompson sampling.
Key results
- On the MNIST bandit, HiLoFi achieves 31,167.7 cumulative reward versus 23,184.5 for LoFi, with pBayes runtime of 8.44 minutes versus 38.90 minutes for TS on LoFi.
- The paper claims 10-100x faster inference than classical Thompson sampling while maintaining comparable or superior decision performance.
- On Bayesian optimization, HiLoFi lies on the Pareto frontier of performance versus compute time, dominated only by VBLL in performance and LRKF in speed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HiLoFi, with full-rank last-layer covariance and low-rank hidden-layer covariance, instead of LoLoFi (ablation) | MNIST contextual bandit with modified LeNet5 · MNIST contextual bandit | cumulative reward (pBayes): 30,038.5 -> 31,167.7 | Helped |
| model the final layer separately rather than use one low-rank covariance for all parameters (LRKF) (ablation) | MNIST contextual bandit · MNIST contextual bandit | cumulative reward (pBayes): 28,354.9 -> 31,167.7 | Helped |
| use posterior predictive sampling instead of Thompson sampling with LoFi | LoFi · MNIST contextual bandit | cumulative reward: 23,304.3 -> 23,184.5 | No clear effect |
| use posterior predictive sampling instead of Thompson sampling with LLL | LLL · MNIST contextual bandit | cumulative reward: 7,201.9 -> 13,922.3 | Helped |
| use posterior predictive sampling instead of Thompson sampling with LoFi | LoFi · MNIST contextual bandit | runtime: 38.90 -> 4.76 | Helped |
| use posterior predictive sampling instead of Thompson sampling with LLL | LLL · MNIST contextual bandit | runtime: 191.55 -> 1.36 | Helped |
| use HiLoFi instead of LRKF | non-stationary Kuairec recommender system · Kuairec | average daily reward | Helped |
| use HiLoFi instead of competing methods | Bayesian optimization benchmarks · Bayesian optimization | performance and compute time | Mixed |
Surprising findings
- LRKF, using a single low-rank covariance for all parameters, performs worse than HiLoFi, which the authors attribute to modeling the last layer separately.
Limitations
- No fixed-lag smoothing.
- Sensitivity to linearization and outliers.
- Reliance on hyperparameters such as process noise.
Experimental setup
Datasets: MNIST KuaiRec Ackley Branin Hartmann DrawNN
Benchmarks: MNIST contextual bandit KuaiRec recommender Bayesian optimization benchmarks
Metrics: cumulative reward daily reward best value found runtime rank
Baselines: Thompson sampling LoFi GP VBLL OnVBLL LLL
Builds on: extended Kalman filter martingale posterior LoFi variational Bayesian last layer Thompson sampling
Closest papers by meaning
- Thompson Sampling via Local Uncertainty (ICML 2020 · similarity 0.87)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.87)
- Bayesian Optimization with Robust Bayesian Neural Networks (NeurIPS 2016 · similarity 0.87)
- Inference and Learning in Dynamic Decision Networks Using Knowledge Compilation (AAAI 2024 · similarity 0.86)
- Scalable Bayesian Learning with posteriors (ICLR 2025 · similarity 0.86)
- Bayesian Optimization via Continual Variational Last Layer Training (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.