Research map / Video / Video-language understanding
GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Grounding
Rong Fan, Kaiyan Xiao, Minghao Zhu, Liuyi Wang, Kai Dai, Zhao Yang
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add query-guided visual token sampling (VTS) on top of grounding fine-tuned model | Qwen2.5VL-7B-G · Charades-STA test | mIoU: 31.7 -> 50.1 | Improved |
| add VTS to grounding fine-tuned model | InternVL3.5-8B-G · Charades-STA test | mIoU: 39.4 -> 41.6 | Improved |
| add VTS | InternVL3.5-8B-G · QVHighlights val (HD) | Hit@1: 39.8 -> 88.4 | Improved |
| add VTS | Qwen2.5VL-7B-G · QVHighlights val (HD) | mAP: 34.4 -> 35.7 | Improved |
| use token-level instead of frame-level query selection (ablation) | GroundVTS-Q · Charades-STA test | mIoU: 41.6 -> 50.1 | Improved |
| use query-guided token sampling instead of uniform (ablation) | GroundVTS-Q · Charades-STA test | mIoU: 29.3 -> 50.1 | Improved |
| use random token dropping instead of uniform frame sampling (ablation) | GroundVTS-Q · Charades-STA test | mIoU: 29.3 -> 35.7 | Improved |
| keep original positional encodings for selected tokens (ablation) | GroundVTS-Q · Charades-STA test | mIoU: 9.5 -> 50.1 | Improved |
| train VTS before use (random-initialized VTS) (ablation) | GroundVTS-Q · Charades-STA test | mIoU: 5.6 -> 20.9 | Improved |
| add stage 2 joint LoRA adaptation (ablation) | GroundVTS-Q · Charades-STA test | mIoU: 32.4 -> 50.1 | Improved |
| add stage 1 VTS warm-up (ablation) | GroundVTS-Q · Charades-STA test | mIoU: 47.4 -> 50.1 | Improved |
| increase frame rate beyond 2.4 FPS (ablation) | Qwen2.5VL-7B (pretrained) · Charades-STA test | mIoU: 47.8 peak at 2.0-2.4 FPS, then sharp drop | Hurt |
Closest papers by meaning
- VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Grounding (AAAI 2025 · similarity 0.92)
- Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Grounding (NeurIPS 2025 · similarity 0.91)
- Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Grounding (NeurIPS 2025 · similarity 0.91)
- TRACE: Temporal Grounding Video LLM via Causal Event Modeling (ICLR 2025 · similarity 0.90)
- VideoGrounding-DINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding (CVPR 2024 · similarity 0.89)
- SnAG: Scalable and Accurate Video Grounding (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
