Research map / Video / Video-language understanding
HanDyVQA: A Video QA Benchmark for Fine-Grained Hand-Object Interaction Dynamics
Masatoshi Tateno, Gido Kato, Hirokatsu Kataoka, Yoichi Sato, Takuma Yagi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune InternVideo2 on the 1.1K training questions (ablation) | InternVideo2-Stage2 · HanDyVQA MCQ | Avg accuracy: 33.3 -> 51.0 | Improved |
| add 3D hand pose input to fine-tuned InternVideo2 (ablation) | InternVideo2-Stage2 fine-tuned · HanDyVQA MCQ | Avg accuracy: 51.0 -> 53.3 | Improved |
| add object features to fine-tuned InternVideo2 (ablation) | InternVideo2-Stage2 fine-tuned · HanDyVQA MCQ | Avg accuracy: 51.0 -> 53.3 | Improved |
| add object bbox tracklets to fine-tuned InternVideo2 (ablation) | InternVideo2-Stage2 fine-tuned · HanDyVQA MCQ | Avg accuracy: 51.0 -> 52.4 | Improved |
| combine all HOI cues (hand + bbox + features) instead of fewer cues (ablation) | InternVideo2-Stage2 fine-tuned · HanDyVQA MCQ | Avg accuracy: Hand+ObjFeats 53.9 vs all three 53.4 | No clear effect |
| increase input frames and resolution (ablation) | Qwen2.5-VL-7B, Gemini-2.5-Pro · HanDyVQA MCQ | accuracy | Improved |
| increase input frames beyond 8 (ablation) | mPLUG-Owl3-8B · HanDyVQA MCQ | accuracy | No clear effect |
| increase input frames and resolution on motion errors (ablation) | MLLMs · HanDyVQA MCQ | Motion error rate | No clear effect |
| use hand/object-aware HelpingHands instead of LaViLa | LaViLa-L (4 frames) · HanDyVQA MCQ zero-shot | Avg accuracy: 41.8 -> 40.3 avg; Objects 67.5 -> 69.2 | Mixed |
| use hand/object-aware EgoHOD instead of LaViLa | LaViLa-L (16 frames) · HanDyVQA MCQ zero-shot | Avg accuracy: Objects 68.5 -> 75.3; Avg 42.6 -> 42.7 | Mixed |
| use video input instead of frame-wise image input for ReasoningVOS (ablation) | Sa2VA-8B · HanDyVQA ReasoningVOS Objects | J: Objects J all 36.2 (image) vs 32.0 (video) | Mixed |
| scale Sa2VA from 4B to 8B (ablation) | Sa2VA · HanDyVQA ReasoningVOS Objects | J (All): 17.2 -> 32.0 | Improved |
Closest papers by meaning
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.89)
- FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation (NeurIPS 2026 · similarity 0.89)
- HandVQA: Diagnosing and Improving Fine-Grained Spatial Reasoning about Hands in Vision-Language Models (CVPR 2026 · similarity 0.89)
- HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awareness (NeurIPS 2024 · similarity 0.89)
- SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios (NeurIPS 2025 · similarity 0.89)
- Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
