Research map / Reinforcement learning / Visual and representation RL
Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Premier-TACO pretrains a multitask visual state representation with a temporal action-driven contrastive loss that samples one hard negative from a temporal window, then fine-tunes few-shot behavior cloning policies on unseen tasks in DMC, MetaWorld, and LIBERO.
Problem
Few-shot policy learning in visual sequential decision-making needs general pretrained representations, but naive multitask contrastive pretraining (TACO with in-batch negatives) is computationally expensive and ineffective when batches mix tasks with different visual appearance.
Approach
Replace TACO's all-in-batch negatives with a single negative sampled from a temporal window around the future state within the same episode, making negatives visually similar to positives so the encoder must capture control-relevant dynamics, and allowing much smaller batch sizes.
Key results
- On 10 unseen DMC tasks with 20 demos, Premier-TACO reaches mean 79.0 vs 47.5 for TACO and 62.4 for SPR (about 101% over learning from scratch).
- On 8 unseen MetaWorld tasks with 5 demos, Premier-TACO reaches 75.8% success vs 55.3 for SPR and 43.6 for learning from scratch.
- On 8 LIBERO long-horizon tasks with 5 demos, Premier-TACO reaches 58.0% success vs 40.3 for learning from scratch and 44.4 for ATC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Premier-TACO pretraining instead of the strongest listed baseline, TD3+BC | few-shot behavior cloning · DMC unseen tasks (10-task mean) | Agent Reward / Expert Reward: 63.3 -> 79.0 | Helped |
| use Premier-TACO pretraining instead of the strongest listed baseline, SPR | five-shot behavior cloning · MetaWorld unseen tasks (8-task mean) | Success rate: 55.3 -> 75.8 | Helped |
| use Premier-TACO pretraining instead of the strongest listed baseline, ATC | five-shot behavior cloning · LIBERO-10 unseen tasks (8-task mean) | Success rate: 44.4 -> 58.0 | Helped |
| use all samples from the size W window as negatives instead of sampling one (ablation) | Premier-TACO · DMC unseen tasks (10-task mean) | Agent Reward / Expert Reward: 79.0 -> 79.8 | No clear effect |
| reduce the pretraining batch size to 512 from 4096 (ablation) | Premier-TACO compared with TACO · MetaWorld | few-shot performance: +25% | Helped |
| vary the negative-sampling window size to W=1 or W=9 instead of W=3, 5, or 7 (ablation) | Premier-TACO · DMC (10 tasks) | average performance | Hurt |
| use randomly collected pretraining trajectories instead of exploratory trajectories (ablation) | Premier-TACO · DMC downstream tasks | few-shot imitation learning performance | No clear effect |
Surprising findings
- Multitask pretraining baselines on LIBERO (multitask BC) degrade downstream few-shot performance relative to learning from scratch.
- Pretraining on uniformly random-action data still gives a substantial gain over learning from scratch on DMC.
Limitations
- Fine-tuning R3M with in-domain data is reported to need the Premier-TACO objective to avoid degradation, so gains depend on the objective choice.
- Evaluation is limited to few-shot behavior cloning on simulated benchmarks with a fixed window size and K=3.
- Pretraining datasets are constructed from specific sources (DrQ-v2 buffers, scripted policies, human demos), so data-quality robustness is only partially tested.
Experimental setup
Datasets: DMC exploratory replay buffers from DrQ-v2 MetaWorld scripted policy with noise LIBERO-90 human demonstrations
Benchmarks: Deepmind Control Suite MetaWorld LIBERO
Metrics: normalized reward (agent/expert) success rate
Baselines: Learn from Scratch SMART Best PVRs TD3+BC Inverse dynamics CURL ATC SPR TACO
Builds on: TACO InfoNCE DrQ-v2 R3M TD3+BC
Closest papers by meaning
- $\texttt{TACO}$: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning (NeurIPS 2023 · similarity 0.92)
- DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors (NeurIPS 2024 · similarity 0.86)
- Task-Conditioned Adaptation of Visual Features in Multi-Task Policy Learning (CVPR 2024 · similarity 0.85)
- Representation Learning Enables Scalable Multitask Deep Reinforcement Learning (NeurIPS 2026 · similarity 0.85)
- Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies (NeurIPS 2025 · similarity 0.85)
- DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.