Research map / Reinforcement learning / Visual and representation RL
Skill Disentanglement in Reproducing Kernel Hilbert Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Hilbert Unsupervised Skill Discovery (HUSD), adding an MMD-based disentanglement reward in an RKHS to entropy-driven skill discovery, and evaluates it on a 2D maze and URLB state-based tasks.
Problem
Unsupervised skill discovery with mutual-information objectives yields either static skills that limit exploration or broad state coverage with non-discriminable skills.
Approach
Adds an intrinsic reward based on the Maximum Mean Discrepancy between the joint distribution of states and skills and the product of their marginals (computed in an RKHS with a Gaussian kernel and incomplete U-statistics), combined with a particle-based state-entropy reward so skills are both separable and cover the state space.
Key results
- HUSD achieves IQM of 79.62% on URLB versus 73.78% for CIC, 73.32% for APT and 70.58% for BeCL.
- HUSD obtains optimality gap of about 22.33%, compared with 27.32% for CIC, 28.10% for APT and 30.98% for BeCL.
- HUSD outperforms CIC on 11 of 12 URLB tasks and beats the other methods on 9 of 12.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the MMD disentanglement reward to the entropy-based reward | HUSD · URLB, 12 downstream tasks | score | Helped |
| Add the MMD disentanglement reward to the entropy-based reward | HUSD · URLB, 12 downstream tasks | IQM score: 79.62% vs 73.78% | Helped |
| Add the MMD disentanglement reward to the entropy-based reward | HUSD · URLB, 12 downstream tasks | optimality gap: 22.33% vs 27.32% | Helped |
| Set λ to zero or a small value instead of using a larger MMD weight (ablation) | HUSD · Square-a maze | skill distinguishability | Hurt |
| Increase λ to [10, 10³] (ablation) | HUSD · Square-a maze | skill distinguishability | Helped |
| Increase λ to [10⁴, 10⁵] (ablation) | HUSD · Square-a maze | state-space coverage | Hurt |
Limitations
- Evaluated only on state-based tasks; extension to pixel-based tasks is left open.
- Performance depends on the weighting parameter lambda, with large values sacrificing state coverage.
- Gains over baselines are modest on several URLB tasks, and the paper does not report compute or code release.
Experimental setup
Datasets: DeepMind Control Suite (URLB) 2D continuous maze
Benchmarks: URLB (Walker, Quadruped, Jaco downstream tasks) 2D continuous maze
Metrics: interquartile mean (IQM) optimality gap (OG) task return
Baselines: ICM Disagreement RND Proto APT SMM DIAYN APS CIC MOSS BeCL
Builds on: Mutual Information Skill Learning CIC APT BeCL URLB
Closest papers by meaning
- Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning (NeurIPS 2024 · similarity 0.92)
- Constrained Ensemble Exploration for Unsupervised Skill Discovery (ICML 2024 · similarity 0.92)
- Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning (ICLR 2024 · similarity 0.90)
- Efficient Skill Discovery via Regret-Aware Optimization (ICML 2025 · similarity 0.90)
- Exploration by Learning Diverse Skills through Successor State Representations (NeurIPS 2024 · similarity 0.89)
- SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.