Research map / Probabilistic and scientific modeling / Dynamical systems
Safely Learning Controlled Stochastic Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a kernel-UCB safe exploration method that iteratively expands a known safe-resettable control set to jointly learn stochastic dynamics densities, safety probabilities, and reset probabilities of controlled SDEs, with safety and Sobolev-adaptive estimation guarantees.
Problem
Learning controlled stochastic dynamics from discrete-time trajectories while keeping all executed trajectories within a predefined safe region with high probability, and enabling safety verification of unseen controls at deployment.
Approach
Extends safe kernelized UCB exploration to jointly learn the state density, safety level, and reset level of a controlled SDE via kernel ridge regression with a Matérn kernel over control-time pairs, expanding the safe-resettable set using LCBs; requires only an initial safe control and Sobolev smoothness of the density.
Key results
- Safety and reset prediction MSE decreases as thresholds relax: safety MSE 0.7010 at (0.1,0.1) vs 0.0023 at unconstrained (+inf).
- Full method runs end-to-end in under 32 minutes on standard hardware for 1000 iterations.
- Theory gives sample complexity N ~ eta^{-2/(1-alpha)} and guarantees all selected controls satisfy s^inf >= 1-eps and r >= 1-xi.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Relax safety and reset thresholds from 0.1 to +∞ (ablation) | Proposed method · 2D stochastic system | Safety MSE: 0.7010 ± 0.3847 -> 0.0023 ± 0.0065 | Helped |
| Relax safety and reset thresholds from 0.1 to +∞ (ablation) | Proposed method · 2D stochastic system | Reset MSE: 0.6919 ± 0.3764 -> 0.0024 ± 0.0062 | Helped |
| Relax safety and reset thresholds (ablation) | Proposed method · 2D stochastic system | Control coverage | Helped |
| Relax safety and reset thresholds (ablation) | Proposed method · 2D stochastic system | Safety | Hurt |
| Run the proposed method end-to-end | Proposed method · 2D stochastic system | Runtime: under 32 minutes | Helped |
Surprising findings
- Stricter safety thresholds create bottlenecks where exploration can stall within the connected safe component, and prediction accuracy is worse for stricter thresholds.
Limitations
- Experiments limited to a low-dimensional 2D synthetic system; scalability to higher dimensions not demonstrated.
- Requires known RKHS norm bounds for confidence parameters, otherwise conservative.
- Sobolev regularity of densities is assumed rather than derived from SDE coefficients.
Experimental setup
Datasets: simulated trajectories from a 2D second-order stochastic dynamical system
Benchmarks: synthetic 2D second-order stochastic dynamical system
Metrics: MSE of safety and reset level prediction control coverage computational runtime
Builds on: safe UCB kernelized bandits (GP-UCB) Matérn kernel
Closest papers by meaning
- A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems (ICML 2025 · similarity 0.89)
- Learning Safe Control via On-the-Fly Bandit Exploration (ICML 2025 · similarity 0.88)
- Verified Safe Reinforcement Learning for Neural Network Dynamic Models (NeurIPS 2024 · similarity 0.88)
- Sampling-Based Safe Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
- Physics-Informed Representation and Learning: Control and Risk Quantification (AAAI 2024 · similarity 0.87)
- Safe and Stable Control via Lyapunov-Guided Diffusion Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.