Research map / Reinforcement learning / Visual and representation RL
Policy Compatible Skill Incremental Learning via Lazy Learning Interface
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SIL-C, a framework that keeps skill-policy compatibility during skill incremental learning by inserting a bilateral lazy-learning interface (Mahalanobis prototype matching with skill validation and hooking) between frozen-policy subtasks and an append-only skill decoder library.
Problem
Skill incremental learning in embodied agents: as the skill library evolves, updated skills can break compatibility with previously trained hierarchical downstream policies (backward compatibility) and fail to be reused by new policies (forward compatibility).
Approach
Reformulate subtask-to-skill alignment as an instance-based (lazy) classification with append-only multimodal Gaussian prototype memories; at inference, a subtask is validated against the predicted subgoal and, if invalid, remapped to the nearest valid skill by trajectory-distribution similarity, requiring no policy retraining or structural change.
Key results
- On Kitchen Emergent SIL, SIL-C (w/ PTGM, AA) reaches 71.8% overall AUC versus 58.2% for PTGM+AA.
- In Kitchen Explicit SIL, SIL-C achieves BwSC BWT of 42.5% versus 0.0% for AA-based Type I baselines... (BwSC BWT of SIL-C 42.5% vs PTGM+AA 0.0%)
- In 1-shot few-shot imitation, SIL-C raises overall AUC from 31.7% to 56.5% over the PTGM baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the SIL-C lazy-learning interface to PTGM + AA | Kitchen Emergent SIL · Overall performance | AUC (%): 58.2 -> 71.8 | Helped |
| Add the SIL-C lazy-learning interface to PTGM + AA | Kitchen Explicit SIL · Overall performance | AUC (%): 36.7 -> 54.1 | Helped |
| Add the SIL-C lazy-learning interface to PTGM | Kitchen Emergent SIL, 5-shot · Overall performance | AUC (%): 49.1% to 62.4% | Helped |
| Add the SIL-C lazy-learning interface to PTGM | Kitchen Emergent SIL, 1-shot · Overall performance | AUC (%): 31.7% to 56.5% | Helped |
| Reduce sampled demonstration transitions from 100% to 20% (ablation) | SIL-C (PTGM), Kitchen Emergent SIL, 1-shot · Overall performance | AUC (%): 56.5 -> 49.7 | Hurt |
| Reduce sampled demonstration transitions from 100% to 20% (ablation) | PTGM, Kitchen Emergent SIL, 1-shot · Overall performance | AUC (%): 31.7 -> 27.0 | Hurt |
| Increase evaluation input noise from ×1 to ×5 (ablation) | SIL-C (PTGM), Kitchen Emergent SIL · Overall performance | AUC (%): 71.8 -> 22.9 | Hurt |
| Increase evaluation input noise from ×1 to ×5 (ablation) | PTGM + AA, Kitchen Emergent SIL · Overall performance | AUC (%): 58.2 -> 18.2 | Hurt |
| Use 40 rather than 10 skill groups per phase (ablation) | SIL-C (PTGM + AA), Kitchen Emergent SIL with 40 subtask groups · Overall performance | AUC (%): 61.5% to 75.5% | Helped |
| Increase subtask groups from 10 to 40 with 10 skill groups (ablation) | SIL-C (PTGM + AA), Kitchen Emergent SIL · Overall performance | AUC (%): 61.5% to 61.4% | No clear effect |
| Increase subtask groups from 10 to 40 with 40 skill groups (ablation) | SIL-C (PTGM + AA), Kitchen Emergent SIL · Overall performance | AUC (%): 73.8% to 75.5% | Helped |
| Replace the learned high-level policy with a random policy (ablation) | SIL-C interface, Kitchen Emergent SIL · Per-phase performance | normalized reward | Hurt |
Surprising findings
- Type II methods with predefined semantic skill labels show near-zero BWT in Explicit SIL, merely preserving initial performance without adaptation.
Limitations
- Relies on unsupervised clustering for skill and subtask spaces, which may degrade with more diverse or noisy skill distributions.
- Evaluated only in simulation with well-defined state representations.
- Near-oracle setting in Emergent Kitchen makes BwSC gains partly reflect favorable conditions.
Experimental setup
Datasets: kitchen-mixed-v0 Meta-World Easy (multi-stage)
Benchmarks: Franka Kitchen Multi-stage Meta-World
Metrics: normalized reward forward transfer backward transfer AUC
Baselines: BUDS (FT) BUDS (ER) BUDS (AA) PTGM (FT) PTGM (ER) PTGM (AA) Skill-prototypes Instructions
Builds on: PTGM BUDS Experience replay Adapter append
Closest papers by meaning
- Robust Policy Learning via Offline Skill Diffusion (AAAI 2024 · similarity 0.88)
- In-Context Policy Adaptation via Cross-Domain Skill Diffusion (AAAI 2025 · similarity 0.88)
- Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning (ICML 2025 · similarity 0.88)
- SkillOS: Learning Skill Curation for Self-Evolving Agents (NeurIPS 2026 · similarity 0.87)
- Incremental Learning of Retrievable Skills For Efficient Continual Task Adaptation (NeurIPS 2024 · similarity 0.87)
- Dynamic Contrastive Skill Learning with State-Transition Based Skill Clustering and Dynamic Length Adjustment (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.