Research map / Reinforcement learning / Visual and representation RL

TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Tasks

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Hierarchical skills and priors · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TGPO, a hierarchical model-free RL method that grounds STL time variables via critic-guided Metropolis-Hastings sampling and trains a time-conditioned PPO policy with dense stage-wise rewards for general nested STL tasks.

Problem

Learning control policies for long-horizon, nested Signal Temporal Logic (STL) tasks is hard for RL because STL is non-Markovian and its robustness reward is sparse, and prior RL methods only handle limited STL fragments.

Approach

Decompose STL into timed subgoals and invariant constraints, ground the free time variables by sampling them with Metropolis-Hastings guided by the learned PPO critic (plus uniform and elite sampling), and train a time-conditioned policy on an augmented MDP with dense stage-wise rewards.

Key results

• TGPO* reaches 86.46% success on Quadrotor and 61.57% on Ant, where most baselines are below 10%.
• On the Linear ablation, the full TGPO* achieves 88.99% test success versus 80.33% for random-sampling-only TGPO.
• The abstract claims an average 31.6% improvement in success rate over the best baseline.

Surprising findings

• Naively using Bayesian (critic-guided) sampling alone hurts performance (53.79% vs 80.33% with random sampling), and needs mixing with random and elite sampling.
• Using only the STL robustness score as reward yields just 11.73% success, showing dense shaping is essential.

Limitations

• No formal convergence guarantee to a global optimum.
• Does not handle disjunctions or G(F) infinite-horizon specifications efficiently.
• Only tested with up to 5 time variables; scalability to more complex STLs is open.

Experimental setup

Datasets: simulated environments (Linear, Unicycle, Franka Panda, Quadrotor, Ant)

Benchmarks: Linear Unicycle Franka Panda Quadrotor Ant

Metrics: task success rate training time

Baselines: RNN CEM Grad tau-MDP F-MDP

Builds on: PPO tau-MDP F-MDP Cross-Entropy Method

Closest papers by meaning

• One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
• Zero-Shot Trajectory Planning for Signal Temporal Logic Tasks (NeurIPS 2025 · similarity 0.87)
• TeLoGraF: Temporal Logic Planning via Graph-encoded Flow Matching (ICML 2025 · similarity 0.87)
• Absolute State-wise Constrained Policy Optimization: High-Probability State-wise Constraints Satisfaction (NeurIPS 2026 · similarity 0.87)
• Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control (ICLR 2025 · similarity 0.87)
• Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning (ICML 2024 · similarity 0.86)