Research map / Vision-language and multimodal / Multimodal LLMs
Spatial and Visual Perspective-Taking via View Rotation and Relation Reasoning for Embodied Reference Understanding
Cheng Shi, Sibei Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add depth estimation (ablation) | baseline · YouRefIt test | Prec@0.50: 39.0 -> 40.8 | Improved |
| add embodied 3D coordinate with depth (ablation) | baseline · YouRefIt test | Prec@0.50: 39.0 -> 41.7 | Improved |
| add body language vector (ablation) | REP with embodied coordinate · YouRefIt test | Prec@0.75: 14.5 -> 16.1 | Improved |
| add verbal attention (ablation) | REP without gesture attention · YouRefIt test | Prec@0.50: 42.4 -> 44.0 | Improved |
| add nonverbal gesture attention (ablation) | REP with verbal attention · YouRefIt test | Prec@0.50: 44.0 -> 45.7 | Improved |
| add nonverbal gesture attention (ablation) | REP with verbal attention · YouRefIt test | Prec@0.50 (small objects): 23.2 -> 25.4 | Improved |
| add full 3D view rotation module (ablation) | baseline · YouRefIt test | Prec@0.75: 11.0 -> 16.1 | Improved |
| replace YouRefIt with REP | YouRefIt Full · YouRefIt test | Prec@0.50: 40.5 -> 45.7 | Improved |
| train on ERU data instead of pretraining on REC | ReSC · YouRefIt test | Prec@0.50: 16.3 -> 34.9 | Improved |
| add nonverbal PAF cue (YouRefIt) over ReSC | ReSC · YouRefIt test | Prec@0.50: 34.9 -> 40.5 | Improved |
Closest papers by meaning
- SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs (ICLR 2026 · similarity 0.88)
- Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs (AAAI 2026 · similarity 0.86)
- 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.86)
- SAW-Bench: Learning Situated Awareness in the Real World (ICML 2026 · similarity 0.86)
- ScanERU: Interactive 3D Visual Grounding Based on Embodied Reference Understanding (AAAI 2024 · similarity 0.86)
- SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
