Research map / Large language models / Agents, code and math
A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DLM, which uses an LLM (Gemini Pro) to turn human language policy prompts into reward functions for RMAB resource allocation, refined by reflection on simulated state-feature outcome distributions, and tests it in an ARMMAN-based simulation.
Problem
RMAB resource-allocation policies use fixed reward functions and cannot easily adapt to changing public health priorities, such as prioritizing specific subpopulations.
Approach
Use an LLM as an automated reward designer: it proposes reward functions as code from a language prompt and arm features, trains PPO policies per candidate in a multi-agent RMAB simulator, and reflects on state-feature outcome distributions (no ground-truth reward) to select candidates for in-context refinement.
Key results
- DLM (Reflection) achieves average MNR of 0.93 ± 0.006 versus Base reward (1.0), reaching at least 90% of base performance in 11/16 tasks.
- DLM (No Reflection) averages MNR 0.87 ± 0.008; Default fixed reward averages 0.57 ± 0.027.
- DLM (Reflection) outperforms Default reward in 16/16 prompts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reflection and iterative refinement to DLM (ablation) | DLM · 16 public-health prompts | MNR: 0.87 ± 0.008 -> 0.93 ± 0.006 | Helped |
| use DLM with reflection instead of the fixed Default reward | Public-health resource allocation policies · 16 public-health prompts | MNR: 0.57 ± 0.027 -> 0.93 ± 0.006 | Helped |
| add reflection to zero-shot DLM reward generation (ablation) | DLM · 16 public-health prompts | feature recall | Helped |
| add Chain-of-Thought reasoning (ablation) | DLM · public-health prompts | MNR | No clear effect |
Surprising findings
- Reflection did not help in some tasks where zero-shot No Reflection already achieved high reward, and reflection may degrade performance by iterating unnecessarily.
- Chain-of-thought reasoning showed no significant improvement over the base prompting in the reported comparison.
Limitations
- Purely simulated evaluation; no real-world deployment or field testing.
- Prompts tested only in English; ambiguous prompts can lead to misaligned rewards requiring human disambiguation.
Experimental setup
Datasets: ARMMAN anonymized quality improvement study dataset (January 2022)
Benchmarks: simulated ARMMAN public health RMAB environment (16 prompts)
Metrics: mean normalized reward interquartile mean precision and recall of features
Baselines: Base reward policy Default reward policy Random No Action DLM (No Reflection)
Builds on: Eureka Whittle index PPO Lagrangian relaxation for RMABs
Closest papers by meaning
- PRIORITY2REWARD: Incorporating Healthworker Preferences for Resource Allocation Planning (AAAI 2025 · similarity 0.93)
- Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits (AAAI 2025 · similarity 0.88)
- Optimizing Vital Sign Monitoring in Resource-Constrained Maternal Care: An RL-Based Restless Bandit Approach (AAAI 2025 · similarity 0.86)
- Generator-Mediated Bandits: Thompson Sampling for GenAI-Powered Adaptive Interventions (NeurIPS 2025 · similarity 0.84)
- RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions (NeurIPS 2024 · similarity 0.84)
- Improving Health Information Access in the World’s Largest Maternal Mobile Health Program via Bandit Algorithms (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.