Research map / Reinforcement learning / Policy gradient and actor-critic
Accelerated Learning with Linear Temporal Logic using Differentiable Simulation
Alper Kamil Bozkurt, Calin Belta, Ming Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use differentiable LTL rewards instead of discrete LTL rewards (ablation) | SHAC/AHAC · CartPole, Hopper, Cheetah, Ant LTL tasks | LTL satisfaction probability | Improved |
| use differentiable-simulator RL (SHAC/AHAC) with differentiable LTL rewards vs PPO/SAC | SHAC/AHAC vs PPO/SAC · CartPole | LTL satisfaction probability: near-optimal (Pr>0.8) within 20 M steps vs no meaningful reward after 100 M | Improved |
| use differentiable RL with differentiable LTL rewards vs PPO/SAC | SHAC/AHAC vs PPO/SAC · Hopper | steps to near-optimal: 20 M vs 100 M steps | Improved |
| use differentiable RL with differentiable LTL rewards vs PPO/SAC | SHAC/AHAC vs PPO/SAC · Cheetah and Ant | LTL satisfaction probability | Improved |
| simplify the LTL formula (reachability only) (ablation) | PPO/SAC · CartPole and legged robots | LTL satisfaction probability: to Pr>0.9 | Improved |
| make reward machine differentiable | SHAC · Cheetah RM Task 1 | return (K): 8.3 -> 15.4 at 3000K steps | Improved |
| make reward machine differentiable | SHAC · Cheetah RM Task 2 | return (K): 9.1 -> 21.1 at 3000K steps | Improved |
| use differentiable instead of discrete LTL rewards for PPO/SAC (ablation) | PPO/SAC · all environments | LTL return | No clear effect |
Closest papers by meaning
- DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RL (ICLR 2025 · similarity 0.91)
- Regret-Free Reinforcement Learning for Temporal Logic Specifications (ICML 2025 · similarity 0.90)
- Automaton Constrained Q-Learning (NeurIPS 2025 · similarity 0.89)
- One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Reinforcement Learning with LTL and $\omega$-Regular Objectives via Optimality-Preserving Translation to Average Rewards (NeurIPS 2024 · similarity 0.88)
- Zero-Shot Instruction Following in RL via Structured LTL Representations (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
