Research map / Vision-language and multimodal / Robot action and navigation

STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning

ICLR 2025 · Vision-language and multimodal › Robot action and navigation · approach: Multimodal prompt-driven manipulation · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes STRAP, which retrieves sub-trajectories from a large offline robot dataset using DINOv2/CLIP features and subsequence DTW, then trains a language-conditioned transformer BC policy on the retrieved plus target data.

Problem

Few-shot imitation learning for new robot tasks: generalist multi-task policies underperform specialists due to negative transfer, and existing retrieval methods either need in-domain trained encoders or retrieve whole trajectories or single states, underusing cross-task data.

Approach

Segment small test-time target demos into atomic sub-trajectories using an end-effector velocity heuristic, match them to segments of a large offline corpus with subsequence DTW over off-the-shelf vision foundation model embeddings (DINOv2 or CLIP), and train a policy on the union of target and retrieved data.

Key results

• On LIBERO-10 with best K, STRAP (DINOv2) reaches 58.1% average success versus 51.7% for fine-tuning and 33.4% for BehaviorRetrieval.
• On real-world DROID-Kitchen, STRAP reaches 36.36 (Table), 61.36 (Sink) and 57.12 (Stove) in the Kitchen setting, versus at most 45.00 for the best baseline (MT, Stove).
• Sub-trajectory retrieval improves over full-trajectory DTW retrieval (D-T) by +4.1% averaged across the 10 LIBERO tasks.

Measured effects reported in the paper

Surprising findings

• Full-trajectory retrieval with DINOv2 (D-T) and state-based retrieval with off-the-shelf DINOv2 (D-S) are competitive with or sometimes better than learned encoders on some tasks, and CLIP and DINOv2 perform similarly.
• Multi-task training sometimes ignores the language instruction and executes a task seen in the offline data instead of the target task.

Limitations

• The chunking heuristic can produce overly long sub-trajectories, which hurt performance on Cream-Cheese-Butter.
• Optimal K is highly task-dependent and must be tuned per task.
• Few-shot real-world evaluation is limited to a small number of tasks and scenes.

Experimental setup

Datasets: LIBERO-10 LIBERO-90 DROID DROID-Kitchen (self-collected) Franka-Pen-in-Cup (self-collected)

Benchmarks: LIBERO-10 DROID-Kitchen (real world) Franka-Pen-in-Cup (real world)

Metrics: success rate pick and place partial/full success

Baselines: Behavior Cloning Fine-tuning Multi-task Policy BehaviorRetrieval FlowRetrieval DINOv2 state retrieval DINOv2 full-trajectory retrieval

Builds on: dynamic time warping BehaviorRetrieval FlowRetrieval DROID LIBERO

Closest papers by meaning

• Efficient Robotic Policy Learning via Latent Space Backward Planning (ICML 2025 · similarity 0.89)
• RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches (ICLR 2024 · similarity 0.88)
• Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers (NeurIPS 2024 · similarity 0.88)
• Foundation Policies with Hilbert Representations (ICML 2024 · similarity 0.88)
• Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.88)
• What Matters in Learning from Large-Scale Datasets for Robot Manipulation (ICLR 2025 · similarity 0.87)