Research map / Vision-language and multimodal / Cross-modal learning
TouchFormer: A Robust Transformer-based Framework for Multimodal Material Perception
Kailin Lyu, Long Xiao, Jianing Zeng, Junhao Dong, Xuexin Liu, Zhuojun Zou, Haoyue Yang, Lin Shu, Jie Hao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Modality-Adaptive Gating (MAG) to the fusion baseline (ablation) | TouchFormer baseline · LMTHM SSMC | accuracy (%): 91.32 -> 93.15 | Improved |
| add MAG to the fusion baseline (ablation) | TouchFormer baseline · LMTHM USMC | accuracy (%): 90.17 -> 92.56 | Improved |
| add MAG to the fusion baseline (ablation) | TouchFormer baseline · LMTHM SSMC fine-grained | accuracy (%): 83.53 -> 84.88 | Improved |
| add Cross-Instance Embedding Regularization (CER) on top of MAG (ablation) | TouchFormer · LMTHM SSMC | accuracy (%): 93.15 -> 95.19 | Improved |
| add CER on top of MAG (ablation) | TouchFormer · LMTHM USMC | accuracy (%): 92.56 -> 94.38 | Improved |
| add CER on top of MAG (ablation) | TouchFormer · LMTHM SSMC fine-grained | accuracy (%): 84.88 -> 89.77 | Improved |
| remove one input modality (any of sound, normal force, friction, acceleration) (ablation) | TouchFormer · LMTHM USMC | accuracy (%): 94.38 -> 87.92 (sound removed) | Hurt |
| use TouchFormer instead of the best baseline (non-visual) | SSMC task · LMTHM SSMC | accuracy (%): 92.71 -> 95.19 | Improved |
| add Gaussian noise to modalities at test time (FISHM) | TouchFormer · FISHM SSMC | accuracy (%): 91.47 -> 89.54 | Hurt |
Closest papers by meaning
- AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors (ICLR 2025 · similarity 0.86)
- Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper (NeurIPS 2025 · similarity 0.85)
- Tactile MNIST: Benchmarking Active Tactile Perception (NeurIPS 2026 · similarity 0.84)
- Universal Visuo-Tactile Video Understanding for Embodied Interaction (NeurIPS 2025 · similarity 0.84)
- Touch-R1: Reinforcing Touch Reasoning in MLLMs (NeurIPS 2026 · similarity 0.84)
- EgoTac: In-the-wild Tactile Prediction from Egocentric Vision (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
