Research map / Bandits and online learning / Online convex optimization and games
Infinite-Horizon Distributionally Robust Regret-Optimal Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives an infinite-horizon Wasserstein-2 distributionally robust regret-optimal LTI controller via a saddle-point formulation, computes it with a frequency-domain Frank-Wolfe method, and gives convex H-infinity rational approximations yielding near-optimal state-space controllers.
Problem
Designing causal LTI controllers that minimize worst-case expected regret over a Wasserstein-2 ambiguity set of possibly time-correlated disturbance distributions in the infinite-horizon setting; the optimal controller is non-rational and prior finite-horizon SDP approaches scale poorly with horizon
Approach
Reformulate infinite-horizon DR-RO as a concave-convex saddle-point over the disturbance auto-covariance operator with a Bures-Wasserstein constraint, show the worst-case spectrum has a finite-dimensional parametrization, solve it via Frank-Wolfe in frequency domain, and approximate the worst-case power spectrum by positive rational functions via a convex LMI feasibility problem in H-infinity norm
Key results
- On the AC15 aircraft system, the DR-RO controller achieves lower worst-case expected regret than H2, H-infinity and RO controllers across radii r (e.g., 59.16 at r=0.01 and 302.08 at r=1).
- A degree-3 rational approximation RA(3) closely matches the non-rational DR-RO regret (e.g., 309.85 vs 1307.12 range comparisons; 302.41 vs 302.08 at r=1).
- In time-domain tests the infinite-horizon DR-RO controller outperforms finite-horizon DR-RO and DR-LQR controllers re-applied every 30 steps under white and worst-case noise.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a degree 1 rational approximation instead of the non-rational DR-RO controller (ablation) | DR-RO · AC15, r=0.01, 1, 1.5, 2, 3 | worst-case expected regret: 59.16 -> 60.49; 302.08 -> 33394.74; 488.57 -> 4475.70; 718.20 -> 9351.89; 1307.12 -> 2376.77 | Hurt |
| Use a degree 2 rational approximation instead of the non-rational DR-RO controller (ablation) | DR-RO · AC15, r=0.01, 1, 1.5, 2, 3 | worst-case expected regret: 59.16 -> 59.58; 302.08 -> 303.33; 488.57 -> 491.75; 718.20 -> 723.96; 1307.12 -> 1318.98 | No clear effect |
| Use a degree 3 rational approximation instead of the non-rational DR-RO controller (ablation) | DR-RO · AC15, r=0.01, 1, 1.5, 2, 3 | worst-case expected regret: 59.16 -> 59.57; 302.08 -> 302.41; 488.57 -> 489.49; 718.20 -> 719.72; 1307.12 -> 1309.85 | No clear effect |
| Use the Wasserstein distributionally robust regret-optimal controller rather than H2, H∞, or RO controllers | AC15 control · AC15, varied ambiguity radius r | worst-case expected regret | Helped |
| Use the infinite-horizon DR-RO controller rather than finite-horizon DR-RO and DR-LQR controllers | AC15 control · AC15, white noise and worst-disturbance scenarios | average LQR cost | Helped |
Surprising findings
- Order-1 rational approximation gives a very large regret at r=1 (33394.74 vs about 302 for the non-rational controller), so low-order approximations can be badly off.
Limitations
- Analysis restricted to scalar disturbances (d_w=1) for the rational approximation; multi-input extension left open.
- Assumes full-information setting with state-space parameters known; partially observable case not covered.
- Rational approximation is only enforced on a finite frequency grid, so it is an approximation.
Experimental setup
Benchmarks: AC15 aircraft model REA4 chemical reactor model HE3 helicopter model
Metrics: worst-case expected regret average LQR control cost
Baselines: H2 controller H-infinity controller regret-optimal controller finite-horizon DR-RO DR-LQR rational approximations RA(1), RA(2), RA(3)
Builds on: regret-optimal control H2 control Wasserstein distributionally robust optimization Frank-Wolfe method
Closest papers by meaning
- Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulator (NeurIPS 2018 · similarity 0.88)
- Distributionally Robust Linear Quadratic Control (NeurIPS 2023 · similarity 0.88)
- Naive Exploration is Optimal for Online LQR (ICML 2020 · similarity 0.88)
- Regret Bounds for Episodic Risk-Sensitive Linear Quadratic Regulator (ICLR 2025 · similarity 0.87)
- Rate-Optimal Online Convex Optimization in Adaptive Linear Control (NeurIPS 2022 · similarity 0.87)
- Online Linear Quadratic Control (ICML 2018 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.