Research map / Large language models / Reasoning and chain of thought

TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TempSamp-R1, a GRPO-style reinforcement fine-tuning method for video temporal grounding that mixes in ground-truth off-policy solutions, uses non-linear soft advantage shaping, and trains a hybrid CoT/non-CoT unified model on Qwen2.5-VL-7B.

Problem

Reinforcement fine-tuning of video MLLMs for temporal grounding with GRPO relies on on-policy sampling, which is inefficient and sparse in the large temporal search space, leading to unstable updates and suboptimal convergence.

Approach

Inject ground-truth annotations as an off-policy solution into the GRPO group, and stabilize the resulting skewed advantages with a non-linear asymmetric reward shaping function (log compression above a threshold, exponential expansion below); also train a hybrid CoT/non-CoT single model.

Key results

• On Charades-STA, TempSamp-R1 (Mixed CoT) reaches R1@0.7 of 56.3 and TempSamp-R1 (CoT) reaches 52.9 vs TimeZero 47.9 and iMOVE 45.3.
• On ActivityNet Captions, TempSamp-R1 (CoT) achieves R1@0.5 of 56.0, exceeding TimeZero's 47.3 by 8.7 points.
• With 500 Charades-STA training videos, TempSamp-R1 reaches R1@0.5 of 64.0 vs GRPO 55.3 and SFT 51.4, and trains in 218 min vs GRPO's 338 min.

Measured effects reported in the paper

Surprising findings

• Naively injecting ground-truth rewards into the GRPO group (mixed-policy) degrades performance (R1@0.5 63.0 vs GRPO 68.9) due to skewed advantages.
• Training with a non-CoT epoch followed by a CoT epoch makes the model robust to test-time prompt choice, while CoT-only inference prompts without matching training cause a large drop.

Limitations

• Requires high-quality ground-truth timestamps as off-policy supervision, which may be unavailable in weakly labeled settings.
• Evaluated only on temporal grounding and highlight detection; other video reasoning tasks unexplored.

Experimental setup

Datasets: Charades-STA ActivityNet Captions QVHighlights

Benchmarks: Charades-STA ActivityNet Captions QVHighlights

Metrics: R1@0.3 R1@0.5 R1@0.7 mIoU mAP HIT@1

Baselines: GRPO TimeZero VideoChat-R1 iMOVE TimeChat HawkEye TRACE VideoChat-T UnLoc-L Qwen2.5-VL-7B

Builds on: GRPO DeepSeek-R1 TimeZero VideoChat-R1

Closest papers by meaning

• TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs (NeurIPS 2026 · similarity 0.89)
• Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Models (ICLR 2026 · similarity 0.89)
• On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.89)
• DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO (NeurIPS 2025 · similarity 0.89)
• TTRL: Test-Time Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Video Temporal Grounding (NeurIPS 2025 · similarity 0.88)