Research map / Vision-language and multimodal / Robot action and navigation

VTDexManip: A Dataset and Benchmark for Visual-tactile Pretraining and Dexterous Manipulation with Reinforcement Learning

ICLR 2025 · Vision-language and multimodal › Robot action and navigation · approach: Memory and subgoal-based manipulation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VTDexManip, a visual-tactile human manipulation video dataset (10 daily tasks, 182 objects) and benchmark for pretraining and RL-based dexterous manipulation.

Problem

Human manipulation videos used for robot pretraining are limited to image and language modalities and to simple parallel grippers, lacking tactile signals for dexterous manipulation.

Approach

Collects a vision-tactile dataset of humans performing daily manipulation tasks on 182 objects, and uses it to pretrain visual-tactile representations for dexterous manipulation policies trained with reinforcement learning.

Experimental setup

Datasets: VTDexManip visual-tactile human manipulation dataset

Benchmarks: VTDexManip

Closest papers by meaning

• Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper (NeurIPS 2025 · similarity 0.91)
• Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets (ICLR 2025 · similarity 0.90)
• EgoTac: In-the-wild Tactile Prediction from Egocentric Vision (NeurIPS 2026 · similarity 0.90)
• Scaffolding Dexterous Manipulation with Vision-Language Models (NeurIPS 2025 · similarity 0.89)
• VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception (ICML 2024 · similarity 0.89)
• Enhancing Tactile-based Reinforcement Learning for Robotic Control (NeurIPS 2025 · similarity 0.89)