Research map / Bandits and online learning / Online convex optimization and games
Online Control in Population Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes simplex LDS model and GPC-S, a gradient-based online controller for population dynamics that achieves sublinear regret against mixing-time-constrained linear policies, tested on controlled SIR and replicator dynamics.
Problem
Online control of population dynamics under adversarial perturbations and time-varying costs; existing regret guarantees require strongly stabilizing comparators, which are impossible for stochastic (simplex-constrained) population dynamics.
Approach
Introduce a simplex LDS whose states stay in the probability simplex and a comparator class of linear policies that mix (rather than strongly stabilize); extend disturbance-action GPC with a learned stationary-distribution parameter p and run lazy mirror descent over it.
Key results
- Regret bound of Õ(τ^{7/2} √(dT)) against mixing-time-constrained linear policies for simplex LDS.
- On controlled SIR (T=200, no noise), GPC-S outperforms full-prevention and no-prevention baselines in cumulative cost.
- On the hospital-flow SIR task, GPC-S closely matches the analytically optimal control of prior work.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GPCS instead of always applying full prevention | Controlled SIR · SIR, no perturbations, T=200 | cumulative cost | Helped |
| use GPCS instead of applying no prevention | Controlled SIR · SIR, no perturbations, T=200 | cumulative cost | Helped |
| use GPCS to control hospital flows | Controlled SIR hospital-flow model · Hospital flows, T=100 | trajectory agreement with optimal solution | Helped |
| use GPCS instead of a one-step best-response controller | Replicator dynamics · Replicator dynamics with noisy cost functions | robustness to noisy cost functions | Helped |
Surprising findings
- No linear policy over a simplex LDS can be strongly stable, and linear regret is unavoidable against all linear policies without a mixing constraint.
Limitations
- Requires the controller to observe the perturbation strength gamma_t each step, which may not hold in general.
- Regret bound depends on the mixing time tau and is for a restricted comparator class with fixed control magnitude in [alpha_lb, alpha_ub].
- Experiments are simulations of SIR and replicator dynamics with known dynamics, not real population data.
Experimental setup
Benchmarks: controlled SIR simulation hospital-flow SIR control task
Metrics: cumulative cost regret control trajectory agreement with optimal control
Baselines: always full prevention no prevention one-step best response controller optimal HJB control (Bertsimas et al.)
Builds on: Gradient Perturbation Controller (GPC) online non-stochastic control lazy mirror descent (FTRL) SIR model
Closest papers by meaning
- Evolutionary Game Theory Squared: Evolving Agents in Endogenously Evolving Zero-Sum Games (AAAI 2021 · similarity 0.84)
- Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation (ICLR 2025 · similarity 0.83)
- Reinforcement Learning for Control of Non-Markovian Cellular Population Dynamics (ICLR 2025 · similarity 0.83)
- Evolutionary Prediction Games (NeurIPS 2025 · similarity 0.83)
- Neural Lagrangian Schr\"{o}dinger Bridge: Diffusion Modeling for Population Dynamics (ICLR 2023 · similarity 0.83)
- Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.