Research map / Large language models / Agents, code and math
BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design
Deepro Choudhury, Sinead Williamson, Adam Golinski, Ning Miao, Freddie Bickford Smith, Michael Kirchhof, Yizhe Zhang, Tom Rainforth
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use full EIG with non-deterministic likelihood instead of predictive entropy (ablation) | BED-LLM · 20 Questions | success rate (%): 79 -> 88 (GPT-4o-mini, Animals) | Improved |
| use prior-likelihood pairing instead of data-estimation pairing (ablation) | BED-LLM · 20 Questions | success rate (%): 70 -> 93 (GPT-4o, Animals) | Improved |
| filter hypotheses for consistency with history instead of raw in-context beliefs (ablation) | BED-LLM · 20 Questions | success rate (%): 25 -> 93 (GPT-4o, Animals) | Improved |
| select questions by explicit EIG estimation instead of LLM judgment (ablation) | BED-LLM · 20 Questions | success rate (%): 70 -> 93 (GPT-4o, Animals) | Improved |
| add BED-LLM question selection over direct prompting | LLM questioner · 20 Questions | success rate (%): 45 -> 93 (GPT-4o, Animals) | Improved |
| add BED-LLM over direct prompting across setups | LLM questioner · 20 Questions | success rate (%): +37.4 percentage points on average | Improved |
| add chain-of-thought to prompt-only questioning | Prompt-Only · 20 Questions | success rate (%): 45 -> 62 (GPT-4o, Animals); 10 -> 10 (Llama-3.1-8B, Things) | Mixed |
| use Split (hypothesis-splitting) over Prompt-Only | Prompt-Only · 20 Questions | success rate (%): 45 -> 83 (GPT-4o, Animals) | Improved |
| use Entropy ablation in preference elicitation instead of full EIG (ablation) | BED-LLM · Preference elicitation | mean film rating | No clear effect |
| add BED-LLM in preference elicitation | LLM questioner · Preference elicitation | mean film rating | Improved |
Closest papers by meaning
- A Multi-Agent Conversational Bandit Approach to Online Evaluation and Selection of User-Aligned LLM Responses (AAAI 2026 · similarity 0.88)
- Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models (NeurIPS 2024 · similarity 0.88)
- Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoning (NeurIPS 2024 · similarity 0.87)
- Broaden your SCOPE! Efficient Multi-turn Conversation Planning for LLMs with Semantic Space (ICLR 2025 · similarity 0.87)
- Position: Uncertainty Quantification Needs Reassessment for Large Language Model Agents (ICML 2025 · similarity 0.87)
- Efficient Evaluation of LLM Performance with Statistical Guarantees (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
