Research map / Reinforcement learning / Visual and representation RL
Online Reinforcement Learning for Mixed Policy Scopes
Junzhe Zhang, Elias Bareinboim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use causal Thompson sampling (c-ts*) instead of standard Thompson sampling over all deterministic policies | CAUSAL-TS* · Simulation Experiment 1 (Fig. 1 diagram, 100 random SCMs) | cumulative regret | Improved |
| Use simplified mixed scope (ts*) instead of all deterministic policies (ts) (ablation) | Thompson sampling · Simulation Experiment 1 | cumulative regret | No clear effect |
| Use randomized trials (rct) instead of adaptive learners | online learner · Simulation Experiment 1 | cumulative regret | Hurt |
| Use causal Thompson sampling (c-ts*) instead of ts and ts* | CAUSAL-TS* · Simulation Experiment 2 (cardiovascular diagram, 100 random SCMs) | cumulative regret | Improved |
| Use simplified mixed scope (ts*) instead of all deterministic policies (ts) (ablation) | Thompson sampling · Simulation Experiment 2 | cumulative regret | No clear effect |
| Use randomized trials (rct) | online learner · Simulation Experiment 2 | cumulative regret | Hurt |
| Use causal Thompson sampling (c-ts*) instead of other learners | CAUSAL-TS* · Simulation Experiment 3 (Fig. 2 diagram, 100 random SCMs) | cumulative regret | Improved |
| Prune redundant policy scope S1,2 via graphical conditions (ts* vs ts) (ablation) | Thompson sampling · Simulation Experiment 3 | cumulative regret | Improved |
| Use randomized trials (rct) | online learner · Simulation Experiment 3 | cumulative regret | Hurt |
| Refine mixed scope by removing redundant scope S1,2 (S to S*) | CAUSAL-UCB* · Fig. 2 diagram, regret bound | regret bound | Improved |
| Use minimal reduction C* of c-collection for canonical SCMs | canonical SCM parametrization · Fig. 1a diagram | cardinality of U2: 48 | Improved |
Closest papers by meaning
- Position: Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination (ICML 2024 · similarity 0.83)
- EpiCare: A Reinforcement Learning Benchmark for Dynamic Treatment Regimes (NeurIPS 2024 · similarity 0.83)
- Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies (NeurIPS 2025 · similarity 0.83)
- A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learning (NeurIPS 2025 · similarity 0.82)
- Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design (ICML 2024 · similarity 0.82)
- Mutli-Armed Bandits with Network Interference (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
