Research map / Reinforcement learning / Visual and representation RL
LLM-based Skill Diffusion for Zero-shot Policy Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LDuS, which uses an LLM to turn language contexts into loss functions that guide a sequentially in-painted skill diffusion planner, enabling zero-shot policy adaptation in MetaWorld.
Problem
Language-conditioned skill policies trained on goal-only annotations fail to adapt zero-shot to unseen contexts such as environmental conditions or user requirements (e.g., speed limits, energy use).
Approach
A VAE-based hierarchical skill diffusion planner with sequential in-painting (conditioning on past state-action pairs) is steered at test time by LLM-generated loss functions for the context, with closed-loop LLM self-critique and iterative refinement of guidance frequency.
Key results
- On MetaWorld with language contexts, LDuS reaches 94.60% SR and 63.52 CR versus 76.42% SR and 33.01 CR for Diffuser+Guidance, the strongest baseline.
- On multi-stage MetaWorld, LDuS averages about 41.7% higher SR and 75.5% higher CR than Diffuser+Guidance.
- Ablation: removing loss guidance drops multi-modal-context CR from 63.52 to 0.58 and SR from 93.05% to 0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LDuS instead of Diffuser + Guidance | Diffusion-based policy adaptation · MetaWorld, language context | CR: 33.01 ± 0.52 -> 63.52 ± 4.28 | Helped |
| Use LDuS instead of Diffuser + Guidance | Diffusion-based policy adaptation · Multi-stage MetaWorld, language context | SR (%): 36.67 ± 2.81% -> 84.03 ± 3.67% | Helped |
| Add loss guidance (ablation) | LDuS · MetaWorld, language context | CR: 65.58 ± 5.02 -> 90.38 ± 3.14 | Helped |
| Add loss guidance (ablation) | LDuS · MetaWorld, language context | SR (%): 95.50 ± 0.96% -> 95.36 ± 1.57% | No clear effect |
| Add loss guidance (ablation) | LDuS · MetaWorld, multi-modal context | CR: 0.58 ± 0.10 -> 63.52 ± 4.28 | Helped |
| Add iterative refinement (ablation) | LDuS · MetaWorld, language context | CR: 85.74 ± 4.28 -> 90.38 ± 3.14 | Helped |
| Add iterative refinement (ablation) | LDuS · MetaWorld, multi-modal context | CR: 33.75 ± 0.48 -> 63.52 ± 4.28 | Helped |
| Add sequential in-painting (ablation) | LDuS · Multi-stage MetaWorld | SR and CR | Helped |
| Add sequential in-painting (ablation) | Diffuser · Multi-stage MetaWorld | SR and CR | Helped |
Surprising findings
- Without loss guidance, LDuS achieves similar success rate (95.50%) on language contexts but much lower context reward, indicating SR alone does not capture context alignment.
- Plain Diffuser and LangDT get near-zero context reward on language and multi-modal contexts despite high success rates without context.
Limitations
- Inference time is increased by iterative LLM calls for code generation and refinement.
- Robustness depends on LLM output variability and prompt design.
- Evaluated only on MetaWorld manipulation; complex tasks requiring environment interaction or dexterous control are not covered.
Experimental setup
Datasets: MetaWorld multi-stage MetaWorld
Benchmarks: MetaWorld multi-stage MetaWorld
Metrics: success rate context reward
Baselines: LangDT LISA LCD Diffuser LCD + Guidance Diffuser + Guidance Kinematic-LLM
Builds on: Diffuser LCD loss-guided diffusion (LGD) conditional VAE Kinematic-LLM
Closest papers by meaning
- In-Context Policy Adaptation via Cross-Domain Skill Diffusion (AAAI 2025 · similarity 0.91)
- Robust Policy Learning via Offline Skill Diffusion (AAAI 2024 · similarity 0.89)
- SemTra: A Semantic Skill Translator for Cross-Domain Zero-Shot Policy Adaptation (AAAI 2024 · similarity 0.89)
- GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation (AAAI 2024 · similarity 0.88)
- Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning (ICML 2025 · similarity 0.87)
- Latent Diffusion Planning for Imitation Learning (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.