Research map / Vision-language and multimodal / Robot action and navigation
VICtoR: Learning Hierarchical Vision-Instruction Correlation Rewards for Long-horizon Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces VICtoR, a hierarchical vision-instruction correlation reward model that decomposes long-horizon tasks into stages and motions using GPT-4 task knowledge, trained only on action-free motion-level videos, and shows improved RL success on long-horizon manipulation.
Problem
Learning reward models for long-horizon robotic manipulation from action-free videos and language instructions; existing VIC reward models lack sub-stage awareness, struggle with task complexity, and poorly estimate object states.
Approach
A hierarchical reward model combining an LLM-generated task decomposition (stages, required object states, motions), a CLIP-based stage detector using object-state classification over MDETR crops, and a motion progress evaluator trained with time, motion, and language-frame contrastive losses; the potential function sums preceding motions plus in-motion progress and is used for potential-based rew
Key results
- VICtoR reaches 58.9% success on the 4-motion move-block task versus 0.0% for LIV and LOReL, and 36.8% for the task-level variant.
- On seven multi-stage tasks VICtoR achieves 96.4% on open drawer + pick block versus 35.1% for LOReL, and 70.9% on a 3-stage task versus 35.0% for LOReL.
- Paper reports a 25% average success-rate gain over the best prior VIC baseline and 43% on tasks with more than 3 motions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use hierarchical stage, motion, and progress rewards instead of no reward signals (ablation) | VICtoR · open box then pick blue block | success rate: 0.0% -> 96.4% | Helped |
| add stage reward (ablation) | VICtoR · open box then pick blue block | success rate: 0.0% -> 22.8% | Helped |
| add motion reward to stage reward (ablation) | VICtoR · open box then pick blue block | success rate: 22.8% -> 29.7% | Helped |
| add progress reward to stage and motion rewards (ablation) | VICtoR · open box then pick blue block | success rate: 29.7% -> 96.4% | Helped |
| add language-frame contrastive loss (ablation) | VICtoR Motion Progress Evaluator · open box then pick blue block | success rate: 0.0% -> 96.4% | Helped |
| add motion contrastive loss (ablation) | VICtoR Motion Progress Evaluator · open box then pick blue block | success rate: 61.8% -> 96.4% | Helped |
| add time contrastive loss (ablation) | VICtoR Motion Progress Evaluator · open box then pick blue block | success rate: 94.0% -> 96.4% | Helped |
| use hierarchical VICtoR instead of LOReL | policy learning with VIC rewards · open box + pick block | success rate: 35.1% -> 96.4% | Helped |
| use hierarchical VICtoR instead of LOReL | policy learning with VIC rewards · move block | success rate: 0.0% -> 58.9% | Helped |
Surprising findings
- Trained only on motion-level action-free videos, VICtoR outperforms baselines trained on full-length long-horizon demonstrations.
- The task-level variant of VICtoR already outperforms LIV and LOReL on several tasks, attributed to the training objective.
Limitations
- Cannot generalize to tasks involving unseen motions; only composes seen motions.
- Relies on object state labels and a ground-truth object detector in simulated policy learning.
- Task knowledge generation depends on GPT-4 (reported 96% accuracy on 50 sampled tasks).
Experimental setup
Datasets: simulated CoppeliaSim motion-level demos XSkill
Benchmarks: custom CoppeliaSim long-horizon manipulation tasks XSkill real-world videos
Metrics: success rate potential curve monotonicity
Baselines: Sparse Reward Stage Reward LOReL LIV VICtoR (task)
Builds on: CLIP LOReL LIV MDETR PPO
Closest papers by meaning
- Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning (ICML 2025 · similarity 0.89)
- KISA: A Unified Keyframe Identifier and Skill Annotator for Long-Horizon Robotics Demonstrations (ICML 2024 · similarity 0.87)
- Subtask-Aware Visual Reward Learning from Segmented Demonstrations (ICLR 2025 · similarity 0.87)
- RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks (NeurIPS 2025 · similarity 0.87)
- VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models (NeurIPS 2025 · similarity 0.87)
- Scaffolding Dexterous Manipulation with Vision-Language Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.