Research map / Reinforcement learning / Visual and representation RL

HDCS: Hierarchy Discovery and Critic Shaping for Reinforcement Learning with Automaton Specification

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Hierarchical skills and priors

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HDCS, an RL method for tasks specified by deterministic finite-state automata (DFA) that discovers hierarchy and shapes critics to handle sparse, non-Markovian rewards.

Problem

RL with sparse, non-Markovian rewards specified by DFAs; existing DFA-based RL algorithms face unresolved difficulties (abstract truncated, details unavailable).

Approach

Combines hierarchy discovery over the automaton structure with critic shaping to make DFA-specified tasks tractable for RL. Details beyond the title and truncated abstract are not available.

Closest papers by meaning

• Exploiting Multiple Abstractions in Episodic RL via Reward Shaping (AAAI 2023 · similarity 0.90)
• Direct Preference Optimization for Primitive-Enabled Hierarchical RL: A Bilevel Approach (ICLR 2026 · similarity 0.89)
• Hierarchical Reinforcement Learning with Targeted Causal Interventions (ICML 2025 · similarity 0.88)
• Deep Reinforcement Learning from Hierarchical Preference Design (ICML 2025 · similarity 0.88)
• HYPRL: Reinforcement Learning of Control Policies for Hyperproperties (NeurIPS 2025 · similarity 0.88)
• Reward Shaping for Reinforcement Learning with An Assistant Reward Agent (ICML 2024 · similarity 0.87)