Research map / Reinforcement learning / Visual and representation RL
Enhancing Tactile-based Reinforcement Learning for Robotic Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes self-supervised auxiliary objectives (tactile reconstruction, full reconstruction, forward dynamics, tactile forward dynamics) and a separated auxiliary memory for on-policy PPO using proprioception and sparse binary tactile contacts, evaluated on the new RoTO benchmark in simulation.
Problem
Tactile sensing in reinforcement learning yields inconsistent gains; sparse, non-smooth tactile signals make it hard for deep RL agents to learn useful representations for dexterous manipulation without idealised state information.
Approach
Jointly train the PPO observation encoder with self-supervised auxiliary losses (tactile reconstruction, multi-step latent forward dynamics with an EMA target encoder) on proprioception plus binary contacts, and decouple the auxiliary training data into a larger separate buffer of recent rollouts.
Key results
- Forward dynamics agent with separated auxiliary memory achieves 17 Baoding rotations in 10 seconds vs 5 for the proprioceptive baseline and 13 for the best RL-only agent (best self-supervised agent reaches 25 in some runs).
- Best Bounce agent achieves 88 bounces in 10 seconds, versus 59 for the human Guinness record figure cited by the authors; FD averages 79 vs 69 bounces.
- Forward dynamics finds the object 36% faster on average (1.4 vs 1.9 s) in Find; TR and FD outperform RL-only baselines across all environments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add binary tactile contacts to proprioceptive history | RL-only PPO · Find | final performance | No clear effect |
| Add binary tactile contacts to proprioceptive history | RL-only PPO · Bounce | evaluation return | Helped |
| Add binary tactile contacts to proprioceptive history | RL-only PPO · Baoding | evaluation return | Helped |
| Remove the last action from proprioceptive observations (ablation) | RL-only PPO · Find | evaluation performance | Hurt |
| Add Tactile Reconstruction (TR) (ablation) | PPO · Find, Bounce, and Baoding | evaluation return | Helped |
| Add Forward Dynamics (FD) (ablation) | PPO · Find, Bounce, and Baoding | evaluation return | Helped |
| Use FD instead of TR (ablation) | PPO with self-supervision · Find and Bounce | mean evaluation return | Helped |
| Use TR instead of FD (ablation) | PPO with self-supervision · Baoding | mean evaluation return | Mixed |
| Add Full Reconstruction (FR) (ablation) | PPO · Find, Bounce, and Baoding | evaluation return | Mixed |
| Add Tactile Forward Dynamics (TFD) (ablation) | PPO · Find, Bounce, and Baoding | evaluation return | Mixed |
| Add separated auxiliary memory to FD (ablation) | FD agent · Find and Bounce | evaluation return | No clear effect |
| Add separated auxiliary memory to FD (ablation) | FD agent · Baoding | evaluation return | Helped |
Surprising findings
- Proprioceptive-only agent with last action succeeds in Find by implicitly inferring contact from control error, and tactile information adds only marginal gains there.
- Tactile reconstruction combined with proprioceptive reconstruction (FR) causes negative interference in Baoding, while TR alone avoids failure runs.
Limitations
- No validation on physical robot hardware; results are simulation-only in Isaac Lab.
- Self-supervised training increases computation time, especially for large forward-dynamics sequence lengths.
- Separated auxiliary memory requires more memory, and the re-sweep over memory size was limited by compute.
Experimental setup
Benchmarks: Find Bounce Baoding
Metrics: mean evaluation return bounces in 10 seconds Baoding rotations in 10 seconds time to find object mutual information between latent and ground-truth state (KS
Baselines: PPO (proprioception only) PPO (proprioception + tactile) full reconstruction auxiliary loss
Builds on: PPO SKRL Isaac Lab Optuna
Closest papers by meaning
- Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper (NeurIPS 2025 · similarity 0.90)
- Multi-fingered Hand Grasps with Visuo-Tactile Fusion via Multi-Agent Deep Reinforcement Learning (AAAI 2025 · similarity 0.89)
- Cross-Embodiment Dexterous Grasping with Reinforcement Learning (ICLR 2025 · similarity 0.87)
- Tactile MNIST: Benchmarking Active Tactile Perception (NeurIPS 2026 · similarity 0.87)
- EgoTac: In-the-wild Tactile Prediction from Egocentric Vision (NeurIPS 2026 · similarity 0.87)
- Touch-R1: Reinforcing Touch Reasoning in MLLMs (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.