Research map / Time series / Causal and generative time series

PAC-Bayes Generalisation Bounds for Dynamical Systems including Stable RNNs

AAAI 2024 · Time series › Causal and generative time series · approach: Dynamical system reconstruction · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives a Catoni-style PAC-Bayes generalisation bound for stable (exponentially convergent) non-linear dynamical systems, including stable RNNs, trained on a single non-i.i.d. time series; the bound does not grow with inference length.

Problem

Generalisation guarantees for time-series prediction with dynamical systems and RNNs learned from a single non-i.i.d. trajectory; existing RNN bounds grow with the number of time steps and assume noiseless models and i.i.d. samples.

Approach

Treats predictors as uniformly exponentially convergent state-space systems with Lipschitz outputs and defines generalisation loss on the infinite horizon; combines Donsker-Varadhan PAC-Bayes inequalities with weak-dependence (theta-mixing) concentration, yielding a bound that depends on mixing coefficients, data magnitude and robustness constants but not on sequence length.

Key results

• The bound is non-vacuous (below trivial error of 1) for N >= 9 in the illustrative RNN example, and converges at rate O(1/sqrt(N)).
• The authors claim the bound is much tighter than the Alquier-Wintenberger linear state-space bound while requiring less knowledge of the generator.

Measured effects reported in the paper

Limitations

• Bound constants (system robustness, mixing coefficients) must be known or estimated and can be conservative, especially for deep RNNs.
• Requires stability (UEC / contraction-type) assumptions on predictors, restricting the model class.
• Only a single small synthetic illustrative experiment is reported.

Experimental setup

Datasets: synthetic time series from a randomly chosen stable ReLU/tanh state-space generator

Benchmarks: synthetic state-space example

Metrics: generalisation bound value vs N empirical loss

Baselines: Alquier and Wintenberger linear state-space PAC-Bayes bound

Builds on: Catoni-style PAC-Bayes bound Donsker-Varadhan change of measure weak dependence / Bernoulli shift theory

Closest papers by meaning

• PAC-Bayesian Error Bound, via Rényi Divergence, for a Class of Linear Time-Invariant State-Space Models (ICML 2024 · similarity 0.89)
• Understanding Generalization in Recurrent Neural Networks (ICLR 2020 · similarity 0.88)
• A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks (ICLR 2018 · similarity 0.87)
• A PAC-Bayesian Approach to Generalization Bounds for Graph Neural Networks (ICLR 2021 · similarity 0.87)
• Controlling Multiple Errors Simultaneously with a PAC-Bayes Bound (NeurIPS 2024 · similarity 0.86)
• Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworks (ICML 2025 · similarity 0.85)