Research map / Vision-language and multimodal / Robot action and navigation
DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DecisionNCE, an InfoNCE-style objective derived from the Bradley-Terry preference model that pretrains vision-language encoders on human videos for robot representation learning and zero-shot rewards.
Problem
Learning embodied multimodal representations that jointly capture local and global task progression, temporal consistency, and trajectory-level language grounding, which existing methods handle with separate objectives.
Approach
Treats video-instruction pairs as implicit preferences (matched segments preferred over mismatched ones), reparameterizes Bradley-Terry rewards in embedding space (potential-based and transition-direction variants), and uses random segment sampling so one InfoNCE-style loss yields grounding and implicit time contrastive learning.
Key results
- On FrankaKitchen MPPI zero-shot planning, DecisionNCE-T achieves 0.122 average success rate vs 0.054 for LIV, 0.036 for R3M, and 0.016 for CLIP.
- DecisionNCE significantly outperforms baselines across almost all of 9 real-robot LCBC tasks, with LIV and R3M the strongest baselines.
- Training takes about 9 hours on four A100 GPUs for 20K steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DecisionNCE-P instead of CLIP for zero-shot implicit-reward MPPI planning | FrankaKitchen · FrankaKitchen | average success rate: 0.016 -> 0.101 | Helped |
| Use DecisionNCE-P instead of R3M for zero-shot implicit-reward MPPI planning | FrankaKitchen · FrankaKitchen | average success rate: 0.036 -> 0.101 | Helped |
| Use DecisionNCE-P instead of LIV for zero-shot implicit-reward MPPI planning | FrankaKitchen · FrankaKitchen | average success rate: 0.054 -> 0.101 | Helped |
| Use DecisionNCE-T instead of CLIP for zero-shot implicit-reward MPPI planning | FrankaKitchen · FrankaKitchen | average success rate: 0.016 -> 0.122 | Helped |
| Use DecisionNCE-T instead of R3M for zero-shot implicit-reward MPPI planning | FrankaKitchen · FrankaKitchen | average success rate: 0.036 -> 0.122 | Helped |
| Use DecisionNCE-T instead of LIV for zero-shot implicit-reward MPPI planning | FrankaKitchen · FrankaKitchen | average success rate: 0.054 -> 0.122 | Helped |
| Use DecisionNCE-P instead of LIV for clustering first-image embeddings | Real robot LCBC · Real robot LCBC | first-image embedding similarity: 0.08 ± 0.005 -> 0.44 ± 0.01 | Helped |
| Use DecisionNCE-T instead of LIV for clustering first-image embeddings | Real robot LCBC · Real robot LCBC | first-image embedding similarity: 0.08 ± 0.005 -> 0.96 ± 0.004 | Helped |
| Use DecisionNCE-P instead of LIV for similarity of first-image embeddings to language means | Real robot LCBC · Real robot LCBC | similarity to language embedding means: -0.06 ± 0.02 -> 0.04 ± 0.008 | Helped |
| Use DecisionNCE-T instead of LIV for similarity of first-image embeddings to language means | Real robot LCBC · Real robot LCBC | similarity to language embedding means: -0.06 ± 0.02 -> 0.02 ± 0.01 | Helped |
Surprising findings
- Pushing the first-frame embeddings of other videos toward mismatched instructions yields task-irrelevant first-frame embeddings clustered together (similarity 0.96 for -T), which the authors argue is desirable.
- Including intermediary transitions in the segment reward can harm representation learning and reduce policy success rate.
Limitations
- Instantiated only on CLIP encoders due to limited resources; not scaled to large VLMs.
- Failures on out-of-domain skills (e.g., topple) and dataset-specific task bias from EPIC-KITCHEN-100 annotations.
- Struggles to distinguish mirror tasks such as turn on vs turn off that have near-identical trajectories.
Experimental setup
Datasets: EPIC-KITCHEN-100 Bridgedata-V2 FrankaKitchen
Benchmarks: FrankaKitchen real robot LCBC tasks Bridgedata-V2 zero-shot reward
Metrics: success rate cosine similarity
Baselines: CLIP R3M VIP LIV RT1 LOReL Voltron
Builds on: Bradley-Terry model CLIP VIP R3M LIV
Closest papers by meaning
- Selective Visual Representations Improve Convergence and Generalization for Embodied AI (ICLR 2024 · similarity 0.87)
- TPRU: Advancing Temporal and Procedural Understanding in Large Multimodal Models (ICLR 2026 · similarity 0.87)
- Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control (NeurIPS 2024 · similarity 0.87)
- Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers (NeurIPS 2024 · similarity 0.87)
- What Matters to You? Towards Visual Representation Alignment for Robot Learning (ICLR 2024 · similarity 0.87)
- MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.