Research map / Time series / Causal and generative time series

Mixture of Online and Offline Experts for Non-Stationary Time Series

AAAI 2025 · Time series › Causal and generative time series · approach: Probabilistic and conformal forecasting · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MOOE, which combines static offline experts from past intervals and an online OGD expert via an exponentially weighted meta expert for non-stationary time series, with regret and generalization bounds plus experiments on real-world and synthetic streams.

Problem

Online learning on non-stationary time series where the current online interval's distribution may have appeared in earlier offline intervals; existing online methods either train one expert on all data (misled by drift) or retrain from scratch per interval (unreliable with few labels).

Approach

Maintain offline experts learned (with a regularizer pulling toward the meta expert's weighted average) from completed intervals plus an online OGD expert, and adaptively combine them with an exponentially weighted meta expert, giving regret and generalization bounds tied to the number of experts and interval size.

Key results

• On eight real-world non-stationary datasets, MOOE reaches the top accuracy in every row, e.g. 93.4 vs 87.3 (Condor) on Kitsune and 78.5 vs 73.1 on Usenet; the paper reports a 13.7% average improvement.
• On recurring concept drift (Email list, Spam filtering), MOOE reaches 97.1 accuracy vs Condor's 95.6 and 96.0 vs 95.4; the paper reports an 11.8% accuracy improvement.
• Theory gives regret of sqrt(T ln K) + 6D sqrt(T beta) for the meta expert plus online expert and an O(1/sqrt(T)) generalization bound.

Measured effects reported in the paper

Surprising findings

• The authors state that a better regret from the online optimizer does not translate into fewer samples for the generalization bound, since the bottleneck is not the optimizer.

Limitations

• Theory assumes convex, smooth losses and a linear hypothesis class in a Hilbert space, while the paper notes non-convex problems are not addressed.
• Generalization bound contains a Rademacher term that the authors say cannot be tightened without further assumptions, so sample-complexity gains from better optimizers are not obtained.
• The number of experts K must satisfy a condition relating regret of offline experts to the online expert, which is hard to verify in practice.

Experimental setup

Datasets: ijcnn cod-rna Usenet Weather GasSensor Powersupply Electricity Covertype WESAD Kitsune Email list Spam filtering

Benchmarks: Usenet Weather GasSensor Powersupply Electricity Covertype WESAD Kitsune Email list Spam filtering ijcnn cod-rna

Metrics: accuracy precision recall regret overall mean predictive accuracy

Baselines: NSE DTEL Condor OGD

Builds on: online gradient descent prediction with expert advice Ader (adaptive dynamic regret)

Closest papers by meaning

• Online Time Series Forecasting with Theoretical Guarantees (NeurIPS 2025 · similarity 0.84)
• Filtered not Mixed: Filtering-Based Online Gating for Mixture of Large Language Models (ICLR 2025 · similarity 0.84)
• Online Estimation via Offline Estimation: An Information-Theoretic Framework (NeurIPS 2024 · similarity 0.83)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts (ICML 2025 · similarity 0.83)
• ChronoEpilogi: Scalable Time Series Selection with Multiple Solutions (NeurIPS 2024 · similarity 0.83)
• Online Mixture of Experts: No-Regret Learning for Optimal Collective Decision-Making (NeurIPS 2025 · similarity 0.83)