Research map / Large language models / Agents, code and math
Language Guided Skill Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Language Guided Skill Discovery (LGSD), which uses LLM-generated state descriptions and sentence-embedding distance as a language-based metric within Wasserstein dependency skill discovery to learn semantically diverse, prompt-constrained skills.
Problem
Existing skill discovery methods maximize proxies (state coverage, discriminability) that do not capture semantic diversity of learned skills, and offer limited control over which semantic subspace skills cover.
Approach
Measure state difference as cosine distance between sentence embeddings of LLM-generated descriptions of states (language-distance), use it as a 1-Lipschitz constraint on a latent representation in a Wasserstein dependency measure framework, and steer the semantic subspace via user prompts; a separate network maps language descriptions to skills for zero-shot goal reaching.
Key results
- On the Franka cube-pushing task, LGSD moved the object an average of 0.5m, about five times the second-best algorithm.
- LGSD discovers more diverse states than all five baselines (DIAYN, DADS, LSD, CSD, METRA) in the manipulation task measured by state coverage.
- In Ant downstream AntSingleGoal and AntMultiGoal tasks, high-level controllers on LGSD skills achieve higher returns and better sample efficiency than prior skill discovery methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use language guidance to discover skills | LGSD · FrankaCube manipulation | average object moved distance: 0.5m, which is five times greater than the value achieved by the second-best algorithm | Helped |
| use language guidance to discover skills | LGSD · FrankaCube manipulation | state coverage | Helped |
| use prompts to constrain the skill search space | LGSD · Ant and FrankaCube | prompt-directed skill coverage | Helped |
| use language-guided skills for high-level control | LGSD · AntSingleGoal and AntMultiGoal | return | Helped |
| use language-guided skills for high-level control | LGSD · AntSingleGoal and AntMultiGoal | sample efficiency | Helped |
| infer skills from natural-language goal descriptions using the learned skill inference network | LGSD · FrankaCube natural-language goal following | goal-reaching success | Helped |
Limitations
- Franka agent could not fully cover all skills within the constrained subspace due to exploration difficulty in pushing objects along long trajectories.
- Relies on rule-based state annotation and text-only LLM, without vision-language grounding.
- Operates on state-level descriptions rather than trajectory-level semantics.
Experimental setup
Benchmarks: Isaac Gym Ant locomotion (AntSingleGoal, AntMultiGoal) Franka robot-arm object pushing
Metrics: average object moved distance state coverage (visited 1cm grids) downstream return and sample efficiency
Baselines: DIAYN DADS LSD CSD METRA
Builds on: METRA CSD LSD PPO Sentence-Transformer
Closest papers by meaning
- Language-guided Skill Learning with Temporal Variational Inference (ICML 2024 · similarity 0.89)
- SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions (NeurIPS 2024 · similarity 0.89)
- Periodic Skill Discovery (NeurIPS 2025 · similarity 0.88)
- GROVE: A Generalized Reward for Learning Open-Vocabulary Physical Skill (CVPR 2025 · similarity 0.88)
- Efficient Language-instructed Skill Acquisition via Reward-Policy Co-Evolution (AAAI 2025 · similarity 0.88)
- GoalLadder: Incremental Goal Discovery with Vision-Language Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.