Research map / Video / Video-language understanding
EvoGround: Self-Evolving Video Agents for Video Temporal Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EvoGround, a proposer-solver pair of RL-trained Qwen2.5-VL-7B agents that learn video temporal grounding from 2.5K unlabeled videos, matching or beating supervised baselines on several VTG benchmarks.
Problem
Video temporal grounding normally needs large human-annotated query-moment datasets, which are costly and limit scalability and coverage.
Approach
Two agents initialized from the same VLM backbone co-evolve via RL: a proposer generates query-moment pairs from raw videos, rewarded for format, SigLIP-2 based intra/inter consistency, and solver feedback (conditioned tIoU); a solver is trained with GRPO on these pairs and its accuracy feeds back to the proposer. GDPO is used for the multi-reward proposer.
Key results
- On Charades-STA, EvoGround (2.5K videos) reaches R1@0.5 60.5 and mIoU 53.1, outperforming all SFT-based models and comparable to Time-R1 (R1@0.5 60.8 with human labels).
- On TVGBench, EvoGround achieves R1@0.3 42.1 and mIoU 29.4, surpassing Time-R1* (39.3 and 27.8).
- EvoGround (7.5K, adding 5K Charades-STA raw videos) reaches R1@0.3 81.7 and R1@0.5 67.7 on Charades-STA, with a 4.5-point gain in R1@0.3.
Surprising findings
- Formatting-only proposer rewards yield the worst performance, suggesting collapse to biased, repetitive data without quality constraints.
- Gains on the TVGBench held-out set plateau by iteration 3 when training on only 2.5K videos.
Limitations
- Costly for ultra-long videos because many query-moment pairs need solver feedback.
- Performance depends on the underlying video corpus.
- Gains plateau after 3 iterations with 2.5K videos; out-of-distribution gains from additional in-domain data are modest.
Experimental setup
Datasets: TimeRFT (2.5K raw videos) Charades-STA raw videos (5K, for scaling)
Benchmarks: Charades-STA ActivityNet-Captions TVGBench ReXTime E.T.Bench TemporalBench Video-MME TempCompass
Metrics: R1@0.3 R1@0.5 R1@0.7 mIoU F1 CIDEr BLEU ROUGE sentence similarity SODA_c
Baselines: TimeChat HawkEye VTimeLLM Momentor Qwen2.5-VL Grounded-VideoLLM TimeSuite TRACE ED-VTG VideoChat-R1 VideoChat-R1.5 Time-R1
Builds on: GRPO GDPO Time-R1 DeepSeek-R1 SigLIP-2
Closest papers by meaning
- Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Video Temporal Grounding (NeurIPS 2025 · similarity 0.89)
- Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA (NeurIPS 2026 · similarity 0.87)
- SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agents (NeurIPS 2026 · similarity 0.86)
- SnAG: Scalable and Accurate Video Grounding (CVPR 2024 · similarity 0.86)
- Contextual Self-paced Learning for Weakly Supervised Spatio-Temporal Video Grounding (ICLR 2025 · similarity 0.86)
- Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.