Research map / Reinforcement learning / Offline RL

Dynamic Treatment on Networks

NeurIPS 2026 · Reinforcement learning › Offline RL · approach: Simple practical offline RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Q-Ising, which fits a dynamic Ising model to a single observed network panel, uses the estimated no-intervention adoption probabilities as latent states, and learns bin-level treatment policies with conservative Q-learning, with a finite-sample regret bound.

Problem

Dynamic treatment allocation on a network under interference, where the planner must decide which nodes (bins) to treat and when, using only one observational panel from a historical policy and no experimentation.

Approach

Treat dynamic Ising conditional adoption probabilities (no-intervention baseline) as low-dimensional latent states for offline RL over bin-level actions, with posterior-sampling ensemble policies for uncertainty quantification and a PEVI-style pessimistic regret bound.

Key results

• On the 42 microfinance villages, Q-Ising reports the best or tied-best mean reward in most villages, e.g., village 37: 0.124 vs 0.079 for Degree-Bin.
• Q-Ising's relative improvement over Degree-Bin correlates about -0.5 with village modularity.
• Greedy suboptimality is shown by a 3-node counterexample under SIS dynamics.

Surprising findings

• A random bin policy outperforms most topology-only heuristics on the SBM, because it occasionally seeds the highly susceptible blocks.
• In the microfinance experiments, Plain DQN matches or beats Q-Ising in several villages (e.g., villages 2, 9, 40 region), so Ising augmentation does not uniformly dominate.

Limitations

• Seeds only a single node per period; multi-node budgets left for future work.
• Requires a sufficiently long observational panel for state-action coverage.
• Experiments use simulated SIS dynamics with synthetic heterogeneity rather than real adoption outcomes.

Experimental setup

Datasets: Indian microfinance village networks (Karnataka) synthetic stochastic block model

Benchmarks: Synthetic SBM with SIS dynamics Indian microfinance village networks with simulated SIS dynamics

Metrics: mean period adoption reward percentage improvement over degree-bin

Baselines: Random bin policy Degree centrality Degree-bin LIR Plain DQN

Builds on: conservative Q-learning PEVI dynamic Ising model EMVS LIR

Closest papers by meaning

• Amortized Network Intervention to Steer the Excitatory Point Processes (ICLR 2024 · similarity 0.86)
• Higher-Order Causal Message Passing for Experimentation with Complex Interference (NeurIPS 2024 · similarity 0.84)
• Learning to target with network interference (NeurIPS 2026 · similarity 0.83)
• Off-policy estimation with adaptively collected data: the power of online learning (NeurIPS 2024 · similarity 0.82)
• Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits (AAAI 2025 · similarity 0.82)
• Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect (ICML 2025 · similarity 0.82)