Research map / Vision-language and multimodal / Multimodal LLMs
Beyond Language: Grounding Referring Expressions with Hand Pointing in Egocentric Vision
LING LI, Bowen Liu, Zinuo Zhan, Peng Jie, Jianhui Zhong, Kenglun Chang, Zhidong Deng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| SV-CoT visual chain-of-thought reasoning on Qwen3-VL vs strongest open-source baseline | Qwen3-VL · EgoPoint-Ground EDG (Real-World) | P@0.5: 0.701 -> 0.818 | Improved |
| SV-CoT vs strongest baseline | Qwen3-VL · EgoPoint-Ground EDG (Hybrid) | P@0.5: 0.702 -> 0.756 | Improved |
| SV-CoT keeps accuracy at strict IoU 0.7 threshold | Qwen3-VL · EgoPoint-Ground EDG (Real-World) | P@0.7: 0.616 -> 0.727 | Improved |
| remove hand-pointing cue, language only (D-REC) vs both cues (ablation) | Qwen3-VL-4B · EgoPoint-Ground | P@0.5: 0.699 -> 0.684 | Hurt |
| remove language, pointing only (POG) vs both cues (ablation) | Qwen3-VL-4B · EgoPoint-Ground | P@0.5: 0.699 -> 0.612 | Hurt |
| SV-CoT in language-only setting vs best baseline | Qwen3-VL · EgoPoint-Ground D-REC | P@0.5: 0.684 -> 0.750 | Improved |
| SV-CoT in pointing-only setting vs best baseline | Qwen3-VL · EgoPoint-Ground POG | P@0.5: 0.612 -> 0.639 | Improved |
| real-world vs hybrid evaluation split for baselines | MLLM baselines · EgoPoint-Ground EDG | P@0.5: Qwen3-VL-8B 0.676 -> 0.701; Qwen2.5-VL-3B 0.671 -> 0.632 | Mixed |
| MLLMs on deictic VQA vs human | LLaVA-OneVision-8B · EgoPoint-Ground D-VQA (Real-World) | accuracy: 0.877 (human) vs 0.544 | Hurt |
Closest papers by meaning
- Pointing at Parts: Training-Free Few-Shot Grounding in Multimodal LLMs (CVPR 2026 · similarity 0.89)
- GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation (CVPR 2026 · similarity 0.89)
- DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.89)
- Do Egocentric Video-Language Models Truly Understand Hand-Object Interactions? (ICLR 2025 · similarity 0.88)
- ClawMachine: Learning to Fetch Visual Tokens for Referential Comprehension (ICLR 2025 · similarity 0.88)
- Grounding Multimodal Large Language Models to the World (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
