Research map / Reinforcement learning / Visual and representation RL

Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Unsupervised skill discovery · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Unifies goal-conditioned RL and mutual-information skill learning under a control-maximization view, showing three GCRL formulations are inequivalent and bounding each MISL objective by a formulation-specific goal sensitivity.

Problem

Why skills learned by mutual-information skill learning (MISL) support downstream goal-reaching is theoretically unclear, given that GCRL and MISL each comprise multiple distinct formulations and objectives.

Approach

Defines goal-sensitivity as a measure of control, shows GCRL objectives (persistent goal, exact timing, opportunity window) map to it, and derives information-theoretic bounds linking each formulation to a matching MISL behavioral variable (S_gamma,+, S_K, or a discounted first-visit vector).

Key results

• Proves the three GCRL formulations (OW, Pe, ET) can induce different optimal policies in the same environment.
• For Pe and ET, J(s, pi) = C(s, pi) + 1/N_s, so the maximally in-control policy is optimal for those formulations.
• Goal-behavior MI with S_gamma,+ and S_K is tightly bounded below by a function of goal-sensitivity; for OW, MI with discounted first-visit vector is at least 2C^2.

Surprising findings

• Persistent-goal and exact-timing formulations induce incompatible optimal policies even in the same environment, despite being commonly treated as minor modeling choices.

Limitations

• Assumes finite state, action, and skill spaces.
• Assumes uniform test-time goal distribution (relaxed only in appendix) and deterministic goal-to-skill mapping.
• Bound for opportunity-window formulation is one-directional in full generality.

Experimental setup

Builds on: Klyubin empowerment mutual information skill learning contrastive RL Hindsight Experience Replay

Closest papers by meaning

• Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning (ICLR 2024 · similarity 0.89)
• A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals (ICLR 2025 · similarity 0.89)
• Score Models for Offline Goal-Conditioned Reinforcement Learning (ICLR 2024 · similarity 0.88)
• Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning (ICLR 2025 · similarity 0.88)
• Improving the Continuity of Goal-Achievement Ability via Policy Self-Regularization for Goal-Conditioned Reinforcement Learning (ICML 2025 · similarity 0.88)
• Skill-aware Mutual Information Optimisation for Zero-shot Generalisation in Reinforcement Learning (NeurIPS 2024 · similarity 0.88)