Research map / Reinforcement learning / Visual and representation RL
Dynamic Contrastive Skill Learning with State-Transition Based Skill Clustering and Dynamic Length Adjustment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DCSL, which learns skills from state transitions with a contrastive skill-similarity function and dynamically relabels skill lengths, improving offline skill-based RL on Antmaze, Kitchen, and Meta-World pick-and-place.
Problem
Existing supervised skill-learning methods treat semantically similar action sequences as distinct skills and use fixed skill lengths, which limits flexibility and generalization in long-horizon RL, especially on noisy datasets.
Approach
Represent skills by state transitions rather than action sequences, learn a skill similarity function with an NCE-style contrastive loss to cluster similar behaviors, and relabel skill lengths per data point as the maximal run of consecutive states whose similarity to the skill stays above a threshold.
Key results
- Ours-SAC reaches 94.7% success on Kitchen versus 69.3% for CQL+OPAL and 52.4% for CQL.
- On Antmaze-Large, Ours-SAC reaches 73.7% success versus 70.3% for CQL+OPAL; on Antmaze-Medium CQL+OPAL is higher (81.1% vs 68.0%).
- On Pick-and-Place RP, Ours-SAC needs 64.4 timesteps versus 200.0 for SkiMo-SAC, the lowest among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DCSL skills with SAC instead of SkiMo-SAC | Ant-Medium · Ant-Medium | success rate | Helped |
| Use DCSL skills with CEM instead of SkiMo-CEM | Ant-Medium · Ant-Medium | success rate | Hurt |
| Use DCSL skills with SAC instead of other methods | Ant-Large · Ant-Large | success rate: 70.3 -> 73.7 | Helped |
| Use DCSL skills with SAC instead of SkiMo-SAC | Kitchen · Kitchen | timesteps to complete: 251.3 ± 23.7 -> 165.1 ± 4.4 | Helped |
| Use DCSL skills with SAC instead of other variants | Pick-and-Place RP · Pick-and-Place RP | success rate | Helped |
| Add dynamic skill length relabeling (ablation) | DCSL with CEM · Pick-and-Place MR | success rate | Helped |
| Add dynamic skill length relabeling (ablation) | DCSL · Pick-and-Place ME | success rate | No clear effect |
| Add the skill similarity function (ablation) | DCSL · Pick-and-Place ME | training steps to converge | Helped |
| Add dynamic skill length relabeling and the similarity function (ablation) | DCSL · Pick-and-Place RP | success rate | Helped |
Surprising findings
- Ours-CEM struggles on Antmaze-Medium, which the authors attribute to variable skill lengths destabilizing CEM's planning horizon.
- On Antmaze-Medium, DCSL's success has very high variance (68.0 ± 36.9), and it is outperformed by CQL+OPAL.
Limitations
- Ours-CEM underperforms on Antmaze-Medium due to variable skill lengths conflicting with CEM planning.
- Relies on hand-set hyperparameters such as initial skill length 10, length bounds 4-30, and relabeling threshold 0; no public code is stated.
- Several baselines (CQL+Off-DADS, CQL+OPAL in Kitchen) are missing from some tables because public implementations were unavailable.
Experimental setup
Datasets: D4RL antmaze-medium-diverse D4RL antmaze-large-diverse D4RL kitchen-mixed Meta-World pick-and-place offline datasets (ME, MR, RP)
Benchmarks: D4RL Antmaze-medium D4RL Antmaze-large D4RL Kitchen Meta-World Pick-and-Place Meta-World Peg-Insert-Side
Metrics: success rate timesteps to complete task learning curve return
Baselines: SPiRL SkiMo-SAC SkiMo-CEM BC CQL CQL+Off-DADS CQL+OPAL
Builds on: SPiRL SkiMo Conservative Q-Learning Off-DADS OPAL
Closest papers by meaning
- Robust Policy Learning via Offline Skill Diffusion (AAAI 2024 · similarity 0.89)
- Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning (ICML 2025 · similarity 0.88)
- DIDI: Diffusion-Guided Diversity for Offline Behavioral Generation (ICML 2024 · similarity 0.87)
- Language-guided Skill Learning with Temporal Variational Inference (ICML 2024 · similarity 0.87)
- Policy Compatible Skill Incremental Learning via Lazy Learning Interface (NeurIPS 2025 · similarity 0.87)
- Efficient Skill Discovery via Regret-Aware Optimization (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.