Research map / Probabilistic and scientific modeling / Dynamical systems

Runtime Monitoring of Perception-Based Autonomous Systems via Embedding Temporal Logic

NeurIPS 2026 · Probabilistic and scientific modeling › Dynamical systems · approach: Latent-space control and planning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Embedding Temporal Logic (ETL), a temporal logic whose predicates are distances between observation embeddings and reference-derived target embeddings, with F1-optimal and conformal thresholds, and evaluates runtime monitors on Dubins car, MetaWorld/D3IL manipulation, and DROID.

Problem

Runtime monitoring and formal specification for perception-based autonomous systems is hard because standard temporal logics need low-dimensional state predicates, while learned perception modules needed to produce them are brittle and misaligned with the specification.

Approach

Define temporal logic predicates directly over learned embedding spaces via distances to target embeddings from reference observations, compose them with LTL/STL-style temporal operators, and calibrate thresholds with F1-optimal search or split conformal prediction giving a recall guarantee.

Key results

• On the Dubins car, the F1-optimal ETL monitor reaches F1 0.80-0.85 for atomic specs with agreement above 96%, and conformal thresholds raise reach recall from 0.83 to 0.93.
• On simulated manipulation, ETL (F1-optimal) achieves average F1 0.817 vs 0.790 for logpZO and 0.751 for PCA-kmeans.
• On DROID, ETL reaches mean F1 0.813 and agreement 0.940 versus 0.390 F1 and 0.567 agreement for Qwen2-VL-2B.

Surprising findings

• On MetaWorld pick-place-wall, logpZO (0.992 F1) outperforms ETL (0.961 F1).
• The Qwen2-VL-2B baseline performs competitively on only one of five DROID tasks.

Limitations

• Latent predicates are not fully interpretable in human-understandable terms.
• Monitoring performance depends on whether task-relevant concepts are well separated in the encoder's representation.
• Release predicate on DROID shows low precision (0.162) due to visual ambiguity.

Experimental setup

Datasets: D3IL MetaWorld DROID

Benchmarks: Dubins car navigation D3IL Sorting D3IL Stacking MetaWorld pick-place-wall DROID

Metrics: F1 precision recall agreement sequential ordering accuracy

Baselines: PCA-kmeans logpZO Qwen2-VL-2B

Builds on: signal temporal logic conformal prediction Dreamer

Closest papers by meaning

• STL: Still Tricky Logic (for System Validation, Even When Showing Your Work) (NeurIPS 2024 · similarity 0.86)
• OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation (NeurIPS 2026 · similarity 0.86)
• LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.86)
• Reasoning in Visual Navigation of End-to-end Trained Agents: A Dynamical Systems Approach (CVPR 2025 · similarity 0.85)
• MATA: A Trainable Hierarchical Automaton System for Multi-Agent Visual Reasoning (ICLR 2026 · similarity 0.84)
• Artemis: Structured Visual Reasoning for Perception Policy Learning (ICML 2026 · similarity 0.84)