Research map / Reinforcement learning / Policy gradient and actor-critic
Neural Control and Certificate Repair via Runtime Monitoring
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CertPM and PredPM runtime monitors that check neural control policies together with barrier/Lyapunov certificates in the black-box setting, and uses the detected violations as retraining data to repair policies and certificates (evaluated on SABLAS drone and ship benchmarks).
Problem
Certifying and repairing neural network control policies and certificate functions (barrier and Lyapunov functions) when the system dynamics are unknown (black-box setting).
Approach
Monitor the policy and its certificate jointly at runtime: CertPM flags property or certificate-condition violations on observed states, while PredPM estimates remaining time before violations via a bounded-acceleration optimization and warns in advance; flagged states become new training data for re-training the policy and certificate.
Key results
- On DroneEnv, CertPM repair raised safety rate from 93.99% to 99.13% and BR from 87.03% to 100.00%.
- PredPM with 1000 new violations reached NDR of 91.67% versus 45.38% for the initialized barrier function.
- Repairing a Lyapunov function with CertPM raised the decrease rate on DroneEnv from 83.13% to 91.32%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Repair the policy and barrier function using CertPM-flagged states | SABLAS policy and certificate · DroneEnv | safety rate (SR): 93.99 -> 99.13 | Helped |
| Repair the policy and barrier function using CertPM-flagged states | SABLAS policy and certificate · DroneEnv | barrier rate (BR): 87.03 -> 100.00 | Helped |
| Repair the policy and barrier function using CertPM-flagged states | SABLAS policy and certificate · DroneEnv | non-decreasing rate (NDR): 45.38 -> 90.66 | Helped |
| Replace the simple policy-only monitor with monitoring of both policy and certificate | SABLAS policy repair · DroneEnv | safety rate (SR): 96.61 -> 99.13 | Helped |
| Use PredPM repair with the first threshold configuration | SABLAS policy and certificate · DroneEnv | safety rate (SR): 93.99 -> 99.06 | Helped |
| Use PredPM repair with the second threshold configuration | SABLAS policy and certificate · DroneEnv | safety rate (SR): 93.99 -> 98.67 | Helped |
| Use PredPM repair with the third threshold configuration | SABLAS policy and certificate · DroneEnv | non-decreasing rate (NDR): 45.38 -> 91.67 | Helped |
| Repair the Lyapunov function with CertPM-flagged data | SABLAS policy and Lyapunov function · DroneEnv | decreasing rate (DR): 83.13% to 91.32% | Helped |
Surprising findings
- The simple policy-only monitor baseline gave a safety rate (96.61%) lower than CertPM and PredPM, showing benefit of monitoring certificates too.
- Repair did not reduce policy performance in any observed case, although no guarantees are provided.
Limitations
- Provides no formal correctness guarantees for repaired policies or certificates.
- Repair relies on a well-initialized policy (e.g. SR at least 90%).
- Lyapunov repair required much more data (2.5e4 executions) to detect 3000 violations.
Experimental setup
Datasets: DroneEnv (SABLAS benchmark) ShipEnv (SABLAS benchmark)
Benchmarks: DroneEnv ShipEnv
Metrics: safety rate (SR) barrier condition rate (BR) non-decreasing rate (NDR) number of violations D_New decrease rate (DR) for Lyapunov
Baselines: SABLAS initialized policy simple runtime monitor baseline
Builds on: SABLAS Adam
Closest papers by meaning
- Neural Continuous-Time Supermartingale Certificates (AAAI 2025 · similarity 0.85)
- Certifying Stability of Reinforcement Learning Policies using Generalized Lyapunov Functions (NeurIPS 2025 · similarity 0.85)
- Verified Safe Reinforcement Learning for Neural Network Dynamic Models (NeurIPS 2024 · similarity 0.85)
- Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation (ICML 2024 · similarity 0.85)
- Neural Closure Certificates (AAAI 2024 · similarity 0.84)
- Analytical Lyapunov Function Discovery: An RL-based Generative Approach (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.