Research map / Large language models / Agents, code and math
Language-guided Skill Learning with Temporal Variational Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAST, which uses GPT-4 to propose an initial segmentation of expert trajectories and then a hierarchical temporal variational inference with an MDL auxiliary loss to merge segments into reusable skills for hierarchical RL on BabyAI and ALFRED.
Problem
Discovering reusable skills from unsegmented expert demonstrations for long-horizon tasks, where the segmentation search space grows exponentially with horizon and prior variational methods find poor local optima.
Approach
Use an LLM to produce short (1–5 action) initial segments with language annotations, then learn via temporal variational inference to only merge these segments into skills, with an MDL-based compression objective that accounts for the previous skill when coding switches; language is used to guide skill discovery without restricting skills to language.
Key results
- On BabyAI zero-shot, LAST achieves 43 average success rate vs 37 for LOVE and 36 for LISA and 34 for BC.
- On ALFRED zero-shot subtask evaluation, LAST achieves 52 average success rate vs 47 for LOVE and 46 for LISA.
- In downstream RL on ALFRED, only LAST-based methods achieve positive success on pick clean, pick heat and pick cool tasks with ~200-step horizons.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LAST instead of LOVE | BabyAI · BabyAI zero-shot (Average) | success rate: 37 -> 43 | Helped |
| use LAST instead of LOVE | ALFRED · ALFRED zero-shot (Avg) | success rate: 47 -> 52 | Helped |
| add the MDL auxiliary objective (ablation) | LAST · BabyAI zero-shot | average success rate: 40 -> 43 | Helped |
| remove the LLM-generated initial segmentation (ablation) | LAST · ALFRED downstream tasks | success rate | Hurt |
| remove the constraint that inference only merges initial segments (ablation) | LAST · ALFRED downstream tasks | success rate | Hurt |
| replace LLM segmentation with dataset human instructions (ablation) | LAST · ALFRED downstream tasks | success rate | Hurt |
| remove generated language annotations from the variational inference input (ablation) | LAST · ALFRED downstream tasks | success rate | Mixed |
| remove the MDL auxiliary objective (ablation) | LAST · ALFRED downstream tasks | success rate | Hurt |
| vary the MDL loss weight and resulting skill-library size (ablation) | LAST · ALFRED zero-shot subtasks | average success rate: 47.0; 48.4; 49.6; 52.1; 51.7; 50.0 | Mixed |
Surprising findings
- GPT-4V zero-shot SayCan-style prompting fails to solve any of the six downstream ALFRED tasks.
- Without the initial LLM segmentation, LAST overfits to the short-horizon task (100% on Keychain) but fails on the others.
Limitations
- Requires agents with discrete action tokens so text descriptions of actions can be used by the LLM.
- Number of skills is controlled indirectly via MDL weight and a maximum skill library size; performance is non-monotonic in skill count.
- Uses only symbolic/limited observations for BabyAI and relies on GPT-4 without visual input for segmentation.
Experimental setup
Datasets: BabyAI expert demonstrations ALFRED training dataset
Benchmarks: BabyAI ALFRED
Metrics: task success rate subtask success rate average success rate vs timesteps
Baselines: BC LOVE LISA (SL)^3 seq2seq seq2seq2seq GPT-4V SayCan-style zero-shot
Builds on: LOVE temporal variational inference SAC Gumbel-Softmax
Closest papers by meaning
- Language Guided Skill Discovery (ICLR 2025 · similarity 0.89)
- Learning Parameterized Skills from Demonstrations (NeurIPS 2025 · similarity 0.89)
- Hypothesis, Verification, and Induction: Grounding Large Language Models with Self-Driven Skill Learning (AAAI 2024 · similarity 0.88)
- Dynamic Contrastive Skill Learning with State-Transition Based Skill Clustering and Dynamic Length Adjustment (ICLR 2025 · similarity 0.87)
- GROVE: A Generalized Reward for Learning Open-Vocabulary Physical Skill (CVPR 2025 · similarity 0.87)
- Periodic Skill Discovery (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.