Research map / Vision-language and multimodal / Human and embodied modeling
Understanding Embodied Reference with Touch-Line Transformer
Yang Li, Xiaoxue Chen, Hao Zhao, Jiangtao Gong, Guyue Zhou, Federico Rossano, Yixin Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add VTL prediction and use the virtual touch line as gestural information (ablation) | Touch-Line Transformer · YouRefIt test | precision (IoU=0.75): 37.2 -> 39.0 | Improved |
| Add VTL prediction instead of predicting no gestural line (ablation) | Touch-Line Transformer · YouRefIt test | precision (IoU=0.25): 64.9 -> 71.1 | Improved |
| Add VTL prediction instead of predicting no gestural line (ablation) | Touch-Line Transformer · YouRefIt test | precision (IoU=0.50): 57.4 -> 63.5 | Improved |
| Predict EWL instead of no gestural line (ablation) | Touch-Line Transformer · YouRefIt test | precision across IoU thresholds: 64.9 -> 69.5; 57.4 -> 60.7; 37.2 -> 35.5 | Mixed |
| Replace EWL with VTL prediction (ablation) | Touch-Line Transformer · YouRefIt test | precision across IoU thresholds: 69.5 -> 71.1; 60.7 -> 63.5; 35.5 -> 39.0 | Improved |
| Remove human context from input images using inpainting (ablation) | Touch-Line Transformer · YouRefIt test | precision across IoU thresholds: 64.9 -> 59.1; 57.4 -> 51.3; 37.2 -> 32.4 | Hurt |
| Add the reference alignment loss to encourage co-linearity (ablation) | Touch-Line Transformer with VTL · YouRefIt test | precision across IoU thresholds: 67.1 -> 71.1; 59.0 -> 63.5; 36.4 -> 39.0 | Improved |
| Add the reference alignment loss to encourage co-linearity (ablation) | Touch-Line Transformer with VTL · YouRefIt test | predicted-box cosine similarity: 0.9815 -> 0.9878 | Improved |
| Use VTL rather than EWL as the pointing-line representation | Touch-Line Transformer · YouRefIt test | target-box cosine similarity: 0.9580 -> 0.9901 | Improved |
Closest papers by meaning
- YouRefIt: Embodied Reference Understanding With Language and Gesture (ICCV 2021 · similarity 0.91)
- Ges3ViG : Incorporating Pointing Gestures into Language-Based 3D Visual Grounding for Embodied Reference Understanding (CVPR 2025 · similarity 0.87)
- ScanERU: Interactive 3D Visual Grounding Based on Embodied Reference Understanding (AAAI 2024 · similarity 0.85)
- A Touch, Vision, and Language Dataset for Multimodal Alignment (ICML 2024 · similarity 0.85)
- Pose Priors from Language Models (CVPR 2025 · similarity 0.85)
- Universal Visuo-Tactile Video Understanding for Embodied Interaction (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
