Research map / Reinforcement learning / Policy gradient and actor-critic
Analytical Lyapunov Function Discovery: An RL-based Generative Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a symbolic transformer from scratch with risk-seeking policy gradient, SHGO-based falsification, and GP expert guidance to discover analytical local Lyapunov functions for nonlinear systems up to 10-D.
Problem
Finding valid Lyapunov functions for nonlinear dynamical systems is hard; neural Lyapunov functions suffer from costly formal verification and poor interpretability, while existing analytical approaches target global stability and need costly pre-training datasets.
Approach
An RL-based symbolic transformer is trained from scratch per system: candidates are sampled autoregressively, falsified with SHGO-guided local sampling (counterexamples added to the training set), rewarded by a bounded Lyapunov risk, and updated via risk-seeking policy gradient plus a GP-derived expert guidance loss, with final certification by dReal SMT.
Key results
- Finds valid local Lyapunov functions with 100% success on 2-D to 4-D test systems and 60-100% success on 6-D to 10-D systems over 5 seeds.
- Neural baselines ANLC and FOSSIL 2.0 achieve 0% success on 6-D and 8-D dynamics, while the proposed method achieves 80% on 6-D and 8-D.
- Discovers two previously unreported local Lyapunov functions for a 2-bus (4-D) lossy power system, formally verified with dReal.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply the proposed analytical Lyapunov discovery framework instead of ANLC or FOSSIL 2.0 | Lyapunov discovery · 2-D and 3-D dynamics | success rate: 2-D: 53.3 (ANLC), 80 (FOSSIL 2.0), 100 (Ours); 3-D: 46.7 (ANLC), 66.7 (FOSSIL 2.0), 100 (Ours) | Helped |
| apply the proposed framework instead of neural Lyapunov baselines | Lyapunov discovery · 6-D and 8-D dynamics | success rate: 6-D: 80 (Ours), 0 (ANLC), 0 (FOSSIL 2.0); 8-D: 80 (Ours), 0 (ANLC), 0 (FOSSIL 2.0) | Helped |
| use the proposed method rather than SOS on the 3-D trig system | Lyapunov solving · 3-D trig system | runtime: 6163s vs 157s | Helped |
| use risk-seeking quantile α=0.1 rather than α=0.5 (ablation) | Proposed framework without GP refinement · 3-D trig dynamics | success rate: 33.33% -> 66.67% | Helped |
| use risk-seeking policy gradient with α=0.1 rather than vanilla policy gradient (ablation) | Proposed framework without GP refinement · 3-D trig dynamics | success rate: 0% -> 66.67% | Helped |
| add GP refinement and expert guidance to transformer-only training (ablation) | Proposed framework · 6-D polynomial system | training time | Helped |
| use GP alone rather than transformer-guided GP (ablation) | Lyapunov discovery · 6-D polynomial system | convergence | Hurt |
| blend SHGO-based counterexample feedback with random sampling (ablation) | Verification and training feedback · 6-D polynomial system | training stability and violation detection | Mixed |
| use SMT solver or root-finding rather than SHGO for counterexample feedback (ablation) | Verification and training feedback · 6-D polynomial system | optimization guidance | Hurt |
Surprising findings
- The pre-trained global Lyapunov transformer baseline produced valid functions only on globally stable low-dimensional examples and failed on all locally stable benchmarks.
- Without GP refinement and expert guidance, the risk-seeking ablation at alpha=1 (vanilla policy gradient) achieved 0% success on 3-D trig dynamics.
Limitations
- Relies on a fixed symbolic library and per-system training, so each new dynamics requires a fresh search.
- Local stability only in most cases; global stability requires further expert analysis.
- No completeness or convergence proofs for the overall framework (stated as future work).
Experimental setup
Datasets: hand-collected test dynamical systems (2-D to 10-D)
Benchmarks: 2-D Van der Pol 2-D simple pendulum 3-D trig dynamics 4-D lossy power system 6-D quadrotor 6-D lossless power system 9-D synthetic system 10-D polynomial system
Metrics: success rate over 5 seeds runtime (s) SMT verification time
Baselines: ANLC FOSSIL 2.0 pre-trained transformer global Lyapunov search SOSTOOLS sum-of-squares
Builds on: risk-seeking policy gradient deep symbolic regression SHGO DEAP genetic programming dReal
Closest papers by meaning
- Global Lyapunov functions: a long-standing open problem in mathematics, with symbolic transformers (NeurIPS 2024 · similarity 0.91)
- Certifying Stability of Reinforcement Learning Policies using Generalized Lyapunov Functions (NeurIPS 2025 · similarity 0.89)
- Learning-enabled Polynomial Lyapunov Function Synthesis via High-Accuracy Counterexample-Guided Framework (CVPR 2025 · similarity 0.89)
- Stability Analysis of Switched Linear Systems with Neural Lyapunov Functions (AAAI 2024 · similarity 0.89)
- Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation (ICML 2024 · similarity 0.88)
- Safe and Stable Control via Lyapunov-Guided Diffusion Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.