Research map / Video / Video-language understanding
Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Video Temporal Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes URPA, a GRPO-based test-time adaptation for video temporal grounding that uses rollout-averaged pseudo labels and rollout-variance confidence to weight rewards, adapting from a labelled source to an unlabelled target with only 100–200 target videos.
Problem
Video temporal grounding degrades under domain shift, and GRPO-based grounding needs ground-truth boundaries for rewards, making cross-domain adaptation with unlabelled, large-scale target videos costly and impractical.
Approach
Average GRPO rollout predictions into pseudo boundaries, estimate confidence as exp(-γ·std across rollouts), and scale the tIoU reward by this confidence so that adaptation focuses on reliable pseudo labels; source training uses a relaxed tIoU reward to reduce annotation-boundary bias.
Key results
- On Charades→ActivityNet, URPA with 200-shot target adaptation reaches R@0.5 42.57 versus 35.61 for source-only GRPO on Qwen2.5-7B.
- On TACoS→ActivityNet, URPA (200-shot) reaches R@0.5 41.83 versus 34.82 for source-only GRPO, but remains below full-set UDA-TSL (33.54 R@0.5 is lower, so URPA exceeds it) — URPA reports higher R@0.5 than UDA-TSL on this setting.
- Ablation on Charades→ActivityNet shows removing uncertainty weighting and relaxed tIoU drops R@0.5 from 42.57 to 31.15 for the 200-shot variant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt with 200 rather than 100 unlabelled target videos | URPA · Charades→ActivityNet | R@0.5: 40.13 -> 42.57 | Helped |
| Adapt with 200 rather than 100 unlabelled target videos | URPA · ActivityNet→TACoS | R@0.5: 10.16 -> 10.38 | Helped |
| Adapt with 200 rather than 100 unlabelled target videos | URPA · TACoS→Charades | R@0.7: 30.08 -> 32.04 | Helped |
| Adapt with 200 rather than 100 unlabelled target videos | URPA · TACoS→ActivityNet | R@0.5: 38.41 -> 41.83 | Helped |
| Adapt with 200 rather than 100 unlabelled target videos | URPA · Charades→TACoS | R@0.5: 7.23 -> 8.25 | Helped |
| Adapt with 200 rather than 100 unlabelled target videos | URPA · ActivityNet→Charades | R@0.7: 38.76 -> 39.57 | Helped |
| Add relaxed tIoU pseudo-label processing (ablation) | URPA target adaptation without uncertainty weighting · Charades→ActivityNet | R@0.5: 31.15 -> 41.48 | Helped |
| Add uncertainty-quantified reward weighting (ablation) | URPA with relaxed tIoU pseudo-labels · Charades→ActivityNet | R@0.5: 41.48 -> 42.57 | Helped |
| Add source-domain GRPO training before target adaptation (ablation) | Qwen2.5-7B · Charades→ActivityNet | R@0.5: 1.46 -> 35.61 | Helped |
| Use the proposed source-training reward rather than original GRPO (ablation) | Qwen2.5-7B · Charades→ActivityNet | R@0.5: 35.61 -> 36.46 | Helped |
| Vary uncertainty scaling factor γ across 2, 5, 10, and 25 (ablation) | URPA (100-shot) · Charades→ActivityNet | R@0.5: 40.04; 40.44; 40.13; 39.89 | No clear effect |
| Increase rollout count from 8 to 16, 32, or 64 (ablation) | URPA (200-shot) · Charades→ActivityNet | R@0.5: 42.57 -> 41.78; 41.65; 40.98 | Hurt |
Surprising findings
- Naive pseudo-label adaptation without uncertainty and relaxed tIoU, and the GRPA 100-shot variant on the base model, collapses performance (e.g., R@0.5 1.46 for Qwen2.5-7B + GRPA target adaptation).
Limitations
- Theoretical guarantee relies on strong assumptions (i.i.d. rollouts, bounded second moments) and on GRPO driving KL to zero, which is not verified empirically.
- Results on TACoS→ActivityNet and other settings show only small gains over source-only GRPO for R@0.3/R@0.5 on some targets, and absolute performance on TACoS-target remains low.
Experimental setup
Datasets: ActivityNet Captions TACoS Charades-STA
Benchmarks: ActivityNet Captions TACoS Charades-STA
Metrics: R@0.3 R@0.5 R@0.7 mIoU
Baselines: CBP SCDM CMIN CSMGAN 2DTAN DRN MMN UDA-TSL Qwen2.5-7B GRPO (source only) TimeZero Temporal-R1
Builds on: GRPO Temporal-R1 MC Dropout UDA-TSL
Closest papers by meaning
- Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding (NeurIPS 2025 · similarity 0.90)
- EvoGround: Self-Evolving Video Agents for Video Temporal Grounding (NeurIPS 2026 · similarity 0.89)
- TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs (NeurIPS 2025 · similarity 0.88)
- Universal Video Temporal Grounding with Generative Multi-modal Large Language Models (NeurIPS 2025 · similarity 0.88)
- SnAG: Scalable and Accurate Video Grounding (CVPR 2024 · similarity 0.88)
- Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Grounding (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.