Research map / Reinforcement learning / Visual and representation RL
SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SkiLD, an unsupervised skill discovery method that uses local dependencies between factored state variables as skill targets, learning graph-conditioned skills that induce diverse factor interactions for downstream sparse-reward tasks.
Problem
Diversity-based unsupervised skill discovery in factor-rich environments learns simple skills that only manipulate easy factors and fail on downstream tasks requiring interactions between state factors.
Approach
Represents skills as target state-specific dependency graphs (identified via pCMI with a learned dynamics model) combined with a DIAYN-style diversity indicator; a graph-selection policy with a novelty reward picks under-mastered graphs, and a graph-conditioned skill policy is trained to induce them.
Key results
- SkiLD induces complex interaction graphs (e.g., soaking the rag in the sink, cleaning the car with the rag) that DIAYN and CSD never induce in Mini-BH Cleaning Car.
- SkiLD achieves superior downstream performance on 9 of the 10 evaluated tasks across Mini-BH and iGibson, and is the only method with non-zero success on Clean Rag and Cut Peach.
- Ablation: removing dependency graphs yields zero performance (consistent with DIAYN), while removing the diversity component produces a notable drop, especially on Clean Rag.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SkiLD skills combining local dependency graphs with diversity indicators instead of existing baselines | Downstream tasks across Mini-BH and iGibson · 10 downstream tasks | success rate | Helped |
| Use SkiLD skills instead of baseline methods | Clean Rag and Cut Peach · Clean Rag and Cut Peach | success rate | Helped |
| Add dependency graphs to skill representations (ablation) | SkiLD · Downstream tasks | success rate | Helped |
| Add diversity indicators and diversity reward (ablation) | SkiLD · Downstream tasks, especially Clean Rag | success rate | Helped |
| Use factorized skills, with a separate policy for each state factor, rather than a single monolithic skill policy (ablation) | SkiLD skill learning · Skill learning | learning performance | Helped |
Surprising findings
- Without dependency graphs, the skill-learning method collapses completely to zero downstream performance, matching DIAYN.
Limitations
- Requires a given factored state space; extending to unfactored image observations depends on disentangled representation learning.
- Relies on accurate detection of local dependencies; the learned dynamics model was too hard to use in Thawing and iGibson, so ground-truth dependency graphs were used there.
- Even the best method remains far from mastering the hardest tasks (Clean Rag and Cut Peach).
Experimental setup
Datasets: Mini-behavior Installing Printer Mini-behavior Thawing Mini-behavior Cleaning Car iGibson 2D Minecraft
Benchmarks: Mini-behavior Installing Printer Mini-behavior Thawing Mini-behavior Cleaning Car iGibson 2D Minecraft Mine Gold
Metrics: success rate percentage of episodes inducing each dependency graph
Baselines: DIAYN CSD ELDEN COInS Vanilla RL (PPO)
Builds on: DIAYN CSD ELDEN COInS hindsight experience replay
Closest papers by meaning
- Periodic Skill Discovery (NeurIPS 2025 · similarity 0.89)
- Skill Disentanglement in Reproducing Kernel Hilbert Space (AAAI 2025 · similarity 0.89)
- Language Guided Skill Discovery (ICLR 2025 · similarity 0.89)
- Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Constrained Ensemble Exploration for Unsupervised Skill Discovery (ICML 2024 · similarity 0.88)
- Robust Policy Learning via Offline Skill Diffusion (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.