Research map / Reinforcement learning / Visual and representation RL
In-Context Policy Adaptation via Cross-Domain Skill Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICPAD, a cross-domain skill diffusion framework with domain-agnostic prototype skills, a diffusion skill adapter, and dynamic domain prompting for no-update in-context policy adaptation from few target demonstrations.
Problem
Adapting skill-based RL policies trained in a source domain to long-horizon multi-task target domains with only a few target demonstrations and no model updates or online target interaction.
Approach
Learns domain-agnostic prototype skills jointly with a diffusion-based skill adapter using cross-domain prior and action consistency losses, and adapts at test time by retrieving target-domain prompts via a contrastively trained domain encoder and attention, without updating model weights.
Key results
- On Metaworld under the Low action-noise setting, ICPAD reaches 95.2±4.9% success vs. 77.3±6.2% for DCMRL.
- In CARLA, ICPAD outperforms the strongest baseline DCMRL by 11.6% to 21.6% in normalized return.
- With only 8% of target tasks demonstrated in multi-stage Metaworld, ICPAD drops only 6.8% while DCMRL drops 32.0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ICPAD instead of DCMRL | Cross-domain adaptation policies · CARLA vehicle embodiment domains | normalized return: 11.6% to 21.6% higher normalized returns | Helped |
| use ICPAD instead of DCMRL | Multi-stage Metaworld policies · Wind and action-noise domain changes | success rate: 14.0% higher | Helped |
| use ICPAD instead of DCMRL | Multi-stage Metaworld policies · Horizon length changes | success rate: 0.3% to 1.7% | Helped |
| use ICPAD with 8% of target tasks represented by demonstrations | Multi-stage Metaworld adaptation · Target-task data availability at 8% | success rate: 6.8% decline | Helped |
| add cross-domain action consistency loss to the skill adapter (ablation) | ICPAD · Multi-stage Metaworld, wind domain | success rate: 5.4% to 6.3% | Helped |
| use a diffusion-based skill adapter (ablation) | ICPAD · Multi-stage Metaworld, wind domain | success rate: 5.4% to 6.3% | Helped |
| use retrieval-based dynamic prompting instead of averaging all target data (ablation) | ICPAD · Multi-stage Metaworld, wind domain | success rate: 1.6% to 5.5% | Helped |
| add attention to the retrieved prompts for domain embedding (ablation) | ICPAD · Multi-stage Metaworld, wind domain | success rate: 1.6% to 5.5% | Helped |
Surprising findings
- Retrieval-based dynamic prompting gains grow with domain disparity, while the gap is minimal at low disparity.
Limitations
- Relies on offline multi-domain expert data and few-shot target demonstrations (up to 5 trajectories per task).
- Evaluated only on Metaworld and CARLA simulators; real-world validation not reported.
- Future work noted for multi-modal datasets and semantic alignment across significantly different tasks.
Experimental setup
Datasets: Metaworld offline multi-task data multi-stage Metaworld CARLA driving data
Benchmarks: Metaworld multi-stage Metaworld CARLA
Metrics: success rate normalized return
Baselines: DiffBC+FT SPiRL SPiRL* FIST DCMRL L-DCMRL
Builds on: SPiRL FIST DCMRL DiffBC diffusion models
Closest papers by meaning
- LLM-based Skill Diffusion for Zero-shot Policy Adaptation (NeurIPS 2024 · similarity 0.91)
- Robust Policy Learning via Offline Skill Diffusion (AAAI 2024 · similarity 0.90)
- SemTra: A Semantic Skill Translator for Cross-Domain Zero-Shot Policy Adaptation (AAAI 2024 · similarity 0.89)
- Cross-Domain Policy Adaptation by Capturing Representation Mismatch (ICML 2024 · similarity 0.88)
- Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training (NeurIPS 2025 · similarity 0.88)
- Policy Compatible Skill Incremental Learning via Lazy Learning Interface (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.