Research map / Learning theory and algorithms / Prediction and decision losses
Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AUF-MICNS, an online-ensemble OGD estimator of dynamic influence relations plus a convex QCQP alteration-selection step with increasing-marginal-cost action costs, for avoiding undesired outcomes in non-stationary linear Gaussian environments.
Problem
Avoiding undesired future (AUF) requires choosing low-cost interventions on intermediate variables to keep an outcome in a desired region, but existing rehearsal learning assumes stationary influence relations and ignores action costs and is computationally expensive.
Approach
Track time-varying linear influence parameters with online gradient descent under an online-ensemble over step sizes, and replace the probabilistic constraint with a deterministic chi-square probability-region constraint so the minimal-cost alteration becomes a convex QCQP solvable in polynomial time.
Key results
- On Bermuda data, AUF-MICNS achieves higher success frequency and lower relative alteration cost than QWZ23 and DDPG/PPO/SAC at 100 rounds (exact numbers in figure only).
- Average running time is 2.81 s (Market) and 1.71 s (Bermuda) for MICNS versus 63.14 s and 386.44 s for QWZ23.
- Online-ensemble estimator matches the convergence speed of the best fixed OGD step size without knowing it, with the RL baseline DDPG reaching 0.6955 success frequency only at T=4000 on Market data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AUF-MICNS instead of QWZ23 | Rehearsal-based AUF · Market-managing data | average running time: 63.14 -> 2.81 | Helped |
| Use AUF-MICNS instead of QWZ23 | Rehearsal-based AUF · Bermuda data | average running time: 386.44 -> 1.71 | Helped |
| Use AUF-MICNS instead of the competing methods | AUF decision-making · Market-managing data | success frequency | Helped |
| Use AUF-MICNS instead of the competing methods | AUF decision-making · Bermuda data | success frequency | Helped |
| Use online-ensemble estimation instead of OGD with a fixed step size (ablation) | Influence-relation estimation · Market-managing data | mean-square error | Mixed |
| Use online-ensemble estimation instead of OGD with a fixed step size (ablation) | Influence-relation estimation · Bermuda data | mean-square error | No clear effect |
| Use OGD step size 1.0 instead of 2.5 (ablation) | Influence-relation estimation · Bermuda data | mean-square error | Hurt |
| Use OGD step size 5.0 instead of 2.5 (ablation) | Influence-relation estimation · Bermuda data | mean-square error | Hurt |
| Use a sampled, unknown graph structure instead of a fixed, known graph (ablation) | AUF-MICNS · Bermuda data | success frequency: 0.711 ± 0.018 -> 0.698 ± 0.021 | No clear effect |
Surprising findings
- Model-free RL baselines (DDPG, PPO, SAC) perform poorly in the limited-round setting but DDPG improves with many more rounds (T=4000).
Limitations
- Restricted to linear structural equations; non-linear cases not handled.
- Sensitive to the hyperparameter tau; an inappropriate tau can make the QCQP infeasible.
- Assumes known fixed graph structure and user-specified cost hyperparameters.
Experimental setup
Datasets: Market-managing data (simulated, manually set structural equations) Bermuda ecology dataset (Bermuda environmental variables)
Benchmarks: Market-managing data Bermuda data
Metrics: success frequency relative alteration cost running time mean squared error of estimated parameters
Baselines: QWZ23 DDPG PPO SAC OGD with fixed step sizes
Builds on: rehearsal learning (QWZ23) structural rehearsal model online gradient descent online ensemble
Closest papers by meaning
- Enabling Optimal Decisions in Rehearsal Learning under CARE Condition (ICML 2025 · similarity 0.89)
- Gradient-Based Nonlinear Rehearsal Learning with Multivariate Alterations (AAAI 2025 · similarity 0.86)
- Prospective Learning: Learning for a Dynamic Future (NeurIPS 2024 · similarity 0.85)
- Leaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize (AAAI 2024 · similarity 0.84)
- Cost-Aware Near-Optimal Policy Learning (AAAI 2025 · similarity 0.83)
- Autoregressive Learning in Joint KL: Sharp Oracle Bounds and Lower Bounds (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.