Research map / Vision-language and multimodal / Visual question answering
Can Vision-Language Models Answer Face to Face Questions in the Real-World?
Reza Pourreza, Rishit Dagli, Apratim Bhattacharyya, Sunny Panchal, Guillaume Berger, Roland Memisevic
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio input to VideoLLaMA2.1-7B-AV (pretrained, zero-shot) (ablation) | VideoLLaMA2.1-7B-AV · QIVD | correctness | Hurt |
| fine-tune VideoLLaMA2.1-7B-AV on QIVD with audio and video (5-fold CV) (ablation) | VideoLLaMA2.1-7B-AV · QIVD | correctness: action counting +16.96%, subjective +23.26%, audio-visual +17.39% | Improved |
| fine-tune on QIVD, object attributes and scene understanding (ablation) | VideoLLaMA2.1-7B-AV · QIVD object attributes / scene understanding | correctness: +1.24% / +2.63% | No clear effect |
| fine-tuned model evaluated on video-only input (ablation) | fine-tuned VideoLLaMA2.1-7B-AV · QIVD video-only | correctness | Hurt |
| use fine-tuned Stream-Qwen-Omni to detect when-to-answer instead of Whisper-Streaming | Qwen2.5-Omni-7B · QIVD | when-to-answer MAE (s): 0.83 -> 0.52 | Improved |
| provide accurate when-to-answer timestamp (ablation) | Qwen2.5-Omni-7B · QIVD | correctness | Improved |
| use ground-truth instead of ASR questions and timestamps (ablation) | baseline LMMs · QIVD | correctness: Qwen3-VL-8B 53.72 -> 60.07 | Improved |
| restrict frames to +-0.5 s around question instead of full video (ablation) | Qwen2.5-VL-7B · QIVD | correctness: 60.00 -> 38.28 | Hurt |
| widen frame window to +-3 s around question (ablation) | Qwen2.5-VL-7B · QIVD | correctness: 60.00 -> 61.20 | No clear effect |
| move from static to temporal question categories | all evaluated LMMs · QIVD | correctness: GPT-4o 65.24 -> 46.06 | Hurt |
| scale model to VideoLLaMA2-72B from 7B (ablation) | VideoLLaMA2 · QIVD | correctness: 50.07 -> 50.83 | Improved |
Closest papers by meaning
- OpenEQA: Embodied Question Answering in the Era of Foundation Models (CVPR 2024 · similarity 0.86)
- ECBench: Can Multi-modal Foundation Models Understand the Egocentric World? A Holistic Embodied Cognition Benchmark (CVPR 2025 · similarity 0.85)
- Right this way: Can VLMs Guide Us to See More to Answer Questions? (NeurIPS 2024 · similarity 0.85)
- MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs (NeurIPS 2024 · similarity 0.85)
- Voila-A: Aligning Vision-Language Models with User's Gaze Attention (NeurIPS 2024 · similarity 0.85)
- ChatHuman: Chatting about 3D Humans with Tools (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
