Research map / Probabilistic and scientific modeling / Bayesian optimization
Bayesian Decision Making around Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies Bayesian multi-armed bandits with an expert: pretrains Thompson sampling on expert outcome data and selects between self and expert data via a mutual-information (information-directed) rule, with trust inference for imperfect experts.
Problem
How a Bayesian bandit learner should incorporate expert outcome data (offline or simultaneous) that differs in structure from its own action-outcome experience, and when to trust the expert.
Approach
Treat expert outcomes as Bayesian evidence that prunes parameters inconsistent with the expert's optimal-action outcome distribution; tighten Thompson sampling regret bounds by the mutual information between expert data and the optimal action, and choose which data source to process by maximizing one-step mutual information about the optimal action.
Key results
- In asymmetric bandits, pretraining on 2000 expert samples yields near-zero regret from the start of Thompson sampling, and the information-source selection rule improves regret rate by about 12% and 8% at t=2000 versus single-source agents.
- In strongly asymmetric bandits, 5 expert samples yield zero-regret Thompson sampling in all tested instances.
- In symmetric bandits, offline expert pretraining gives no regret improvement and the MI-based agent consistently chooses its own data.
Surprising findings
- Offline pretraining on expert outcomes gives no benefit in symmetric worlds, where learning from experts only yields linear regret when forced.
Limitations
- Analysis restricted to countable parameter and action spaces with exact posterior updates
- Expert assumed static; simultaneous learning with an evolving expert left to future work
- Only bandit (stateless) settings; not extended to MDPs
Experimental setup
Datasets: synthetic countable bandit worlds (M=500 models, 50 or 20 actions)
Benchmarks: synthetic symmetric bandits synthetic asymmetric bandits synthetic strongly asymmetric bandits
Metrics: cumulative regret rate Reg(T)/T
Baselines: standard Thompson sampling naive trust agent
Builds on: Thompson sampling information-theoretic Bayesian regret analysis (Russo and Van Roy) information-directed sampling
Closest papers by meaning
- Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneity (NeurIPS 2024 · similarity 0.87)
- Principled Bayesian Optimization in Collaboration with Human Experts (NeurIPS 2024 · similarity 0.86)
- Information-Directed Offline-to-Online Reinforcement Learning (NeurIPS 2026 · similarity 0.85)
- Bayesian Strategic Classification (NeurIPS 2024 · similarity 0.85)
- Robustly Improving Bandit Algorithms with Confounded and Selection Biased Offline Data: A Causal Approach (AAAI 2024 · similarity 0.85)
- Online Mixture of Experts: No-Regret Learning for Optimal Collective Decision-Making (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.