Research map / Vision-language and multimodal / Multimodal LLMs

Touch-R1: Reinforcing Touch Reasoning in MLLMs

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces TouchReason-1M (1.3M synchronized optical-tactile pairs across four sensors) and TouchReason-Bench, and trains Touch-R1, a Qwen2.5-VL-7B tactile reasoning MLLM with tactile-grounded GRPO using ordinal, cross-sensor consistency, format, and tactile-dependence rewards.

Problem

Multimodal LLMs struggle to ground physical-property predictions (hardness, roughness, protrusion, material) in optical tactile evidence, especially under visual-tactile conflict, and standard SFT or binary-reward RL does not handle ordinal attributes or cross-sensor shifts.

Approach

Extends GRPO to tactile reasoning with an ordinal-distance-aware accuracy reward, a cross-sensor physical consistency reward, a format reward, and an input-side KL objective that rewards output dependence on the real tactile stream versus a noise-masked version.

Key results

• Touch-R1-7B reaches Avg 60.1 on TouchReason-Bench, versus 47.5 for SToLa and 41.7 for Octopi-13B (the paper reports 18.4% and 24.7% gains over Octopi-13B and GPT-4o on average).
• Ordinal reward lowers OMAE from 0.42 to 0.31 and consistency reward raises CSC from 56.1 to 69.2 in the cumulative ablation.
• Cold-start SFT raises Avg from 28.0 (Qwen2.5-VL-7B zero-shot) to 47.8, and Touch-R1 reaches 65.5 average on VTV150K.

Surprising findings

• Plain GRPO with binary 0/1 accuracy adds only about 2 Avg points over cold-start SFT (47.8 to 50.1), suggesting final-answer supervision alone is insufficient.

Limitations

• Human-labeled ordinal attributes and Gemini-generated rationales may carry annotator and generator biases; only 2,000 QA pairs were human-checked.
• Gains reported over a 4-sensor setup with a fixed interaction protocol; generalization to other sensors or unseen objects beyond the held-out set is not established.

Experimental setup

Datasets: TouchReason-1M Touch-and-Go VTV150K

Benchmarks: TouchReason-Bench Touch-and-Go TouchReason-Video VTV150K

Metrics: H-Acc R-Acc P-Acc Mat-Acc OMAE L2-EM SFD SOI-rho CSC Avg

Baselines: GPT-4o Gemini-2.5-Pro Claude-3.5-Sonnet LLaVA-OneVision-7B Qwen2.5-VL-7B InternVL2.5-8B Octopi-7B Octopi-13B AnyTouch+Qwen VTV-LLM-7B SToLa

Builds on: GRPO DeepSeek-R1 Qwen2.5-VL DAPO Dr. GRPO GSPO

Closest papers by meaning

• STOLA: Self-Adaptive Touch-Language Framework for Tactile Commonsense Reasoning in Open-Ended Scenarios (AAAI 2026 · similarity 0.92)
• AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors (ICLR 2025 · similarity 0.89)
• Universal Visuo-Tactile Video Understanding for Embodied Interaction (NeurIPS 2025 · similarity 0.89)
• Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper (NeurIPS 2025 · similarity 0.89)
• A Touch, Vision, and Language Dataset for Multimodal Alignment (ICML 2024 · similarity 0.88)
• Binding Touch to Everything: Learning Unified Multimodal Tactile Representations (CVPR 2024 · similarity 0.88)