Research map / Vision-language and multimodal / Robot action and navigation
Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Taccel is a GPU-accelerated simulator for vision-based tactile sensors that combines incremental potential contact (IPC) with affine body dynamics (ABD) to simulate robots and soft gel pads, reaching 18.3x real-time with thousands of parallel environments.
Problem
Existing vision-based tactile sensor (VBTS) simulators are either physically imprecise (rigid approximations, penalty contacts) or run below real time with few parallel environments, which limits tactile robotics research at scale.
Approach
Unify IPC contact and affine body dynamics in one GPU framework: robot links are modeled as affine (near-rigid) bodies while gel pads are hyperelastic FEM soft bodies, with a DNN mapping depth and normals to RGB tactile images and marker tracking for low-resolution signals.
Key results
- Peg insertion with 4096 parallel environments reaches 915 FPS (18.30x real-time) on one H100 80GB, versus SAPIEN-IPC's 64-fold lower parallelism.
- Tactile signal sim-real comparison achieves an average SSIM of 0.93 across 18 test objects.
- Object classifier trained only on simulated depth maps reaches 86.50% on synthetic and 70.94% on real test samples without domain adaptation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Taccel instead of SAPIEN-IPC for low-resolution peg insertion | Peg-insertion simulation · Peg insertion, low resolution | simulation speed: 0.81× -> 18.30× | Helped |
| Use Taccel instead of SAPIEN-IPC for high-resolution peg insertion | Peg-insertion simulation · Peg insertion, high resolution | simulation speed: 0.03× -> 0.25× | Helped |
| Use Taccel instead of SAPIEN-IPC for low-resolution peg insertion | Peg-insertion simulation · Peg insertion, low resolution | parallel environments: 256 -> 4096 | Helped |
| Train on synthetic tactile data from Taccel | ResNet-18 mechanical-part classifier · Mechanical part recognition, real-world test set | accuracy: 70.94% | Helped |
| Use Allegro Hand instead of Gripper | Taccel grasping and tactile classification evaluation · Grasped-object classification | accuracy: 44.56% -> 54.30% | Helped |
| Use Allegro Hand instead of F-TAC | Taccel grasping and tactile classification evaluation · Grasped-object classification | accuracy: 42.54% -> 54.30% | Helped |
| Remove ABD from Taccel (ablation) | Taccel · Bolt screwing | simulation capability | Hurt |
| Use Taccel rather than Isaac Sim for Tac-Man articulated manipulation | Tac-Man · Microwave and drawer manipulation | sim-real execution-recovery switch correspondence: 1.10% error | Helped |
Surprising findings
- SAPIEN-IPC's FP32 precision causes convergence issues in contact force solving, which Taccel avoids by using FP64.
Limitations
- Large-scale simulation remains computationally demanding, with PCG-based linear solving as the main bottleneck.
- Dexterous full-hand tactile simulation runs at only 0.25x real-time with 256 environments.
- Sim-to-real gap remains in classification accuracy (86.50% synthetic vs 70.94% real).
Experimental setup
Datasets: ContactDB YCB adversarial object dataset standard tactile shape testing dataset (18 objects)
Benchmarks: peg insertion FPS scaling sim-real tactile SSIM on 18 objects mechanical part classification sim-to-real Tac-Man articulated object switch counts
Metrics: FPS speedup over real-time SSIM classification accuracy friction trajectory error switch count error
Baselines: SAPIEN-IPC Taxim DiffTactile PyBullet Isaac Sim
Builds on: incremental potential contact affine body dynamics SAPIEN-IPC Tac-Man NVIDIA Warp DFC grasp synthesis
Closest papers by meaning
- DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation (ICLR 2024 · similarity 0.90)
- PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation (NeurIPS 2026 · similarity 0.89)
- AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors (ICLR 2025 · similarity 0.88)
- Tactile MNIST: Benchmarking Active Tactile Perception (NeurIPS 2026 · similarity 0.88)
- Enhancing Tactile-based Reinforcement Learning for Robotic Control (NeurIPS 2025 · similarity 0.87)
- EgoTac: In-the-wild Tactile Prediction from Egocentric Vision (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.