Research map / Video / Video-language understanding

EventLens: Event-Structure Reinforcement Learning for Video Understanding

NeurIPS 2026 · Video › Video-language understanding · approach: Long video understanding

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EventLens, a vision-centric RL framework that trains video MLLMs to recover event structure (segmentation, progression direction, multi-event relations) with deterministic rewards via task-specialized GRPO teachers consolidated by multi-teacher on-policy distillation.

Problem

Video MLLMs fail to recover explicit temporal structure (event boundaries, single-event progression, inter-event relations), and text-supervised post-training allows language shortcuts instead of visual temporal grounding.

Approach

Replace caption-style supervision with event-structure recovery tasks under controlled perturbations (granularity, reversed playback, shuffled order) that have deterministic verifiers; train specialized GRPO teachers per task family and merge them into one student with token-level top-k on-policy distillation (MT-OPD).

Key results

• On Qwen3-VL-8B, EventLens reaches 59.12 QVHighlights mIoU, 46.35 ActivityNet mIoU and 48.25 Charades-STA mIoU, the best temporal-grounding scores in its group (base 50.03/39.96/44.79).
• On Qwen3-VL-4B, EventLens improves E.T. Bench average from 34.55 to 41.47 and VinoGround score from 37.00 to 45.00.
• Learning curves reportedly reach comparable transfer after about 13 hours of MT-OPD versus about 37 hours for direct Mixed-GRPO.

Surprising findings

• Caption-style SFT baselines remain competitive or better on several general and reasoning benchmarks (e.g., Video-TT, Video-Holmes), and EventLens-8B trails the base on E.T. Bench among others.
• Gains are not uniform: EventLens drops on montage plot attributes, motion speed, and anomaly recognition at both scales.

Limitations

• Confusion between adjacent events when visual states change gradually or when queries describe high-level activities spanning several actions.
• Visually similar repeated actions remain hard in multi-event relation tasks; long-range narrative reasoning not solved.
• Gains on broad QA are not uniform; some subsets (montage plot attributes, motion speed, anomaly recognition) degrade.

Experimental setup

Datasets: TimeLens-100K ET-Instruct-164K

Benchmarks: QVHighlights ActivityNet Charades-STA Video-Holmes Video-MMMU Video-TT E.T. Bench MVBench TempCompass AoT-Bench VinoGround VideoMME

Metrics: mIoU accuracy official benchmark scores

Baselines: Qwen3-VL-4B Qwen3-VL-8B SFT-Desc SFT-DVC Qwen2.5-VL-7B Video-R1-7B VideoSSR-8B VideoAuto-R1-8B OneThinker-8B

Builds on: GRPO on-policy distillation Qwen3-VL EMA-GRPO VideoSSR

Closest papers by meaning

• ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting (CVPR 2026 · similarity 0.90)
• Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs (NeurIPS 2026 · similarity 0.90)
• E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding (NeurIPS 2024 · similarity 0.88)
• Enhancing Temporal Understanding in Video-LLMs through Stacked Temporal Attention in Vision Encoders (NeurIPS 2025 · similarity 0.88)
• VTimeLLM: Empower LLM to Grasp Video Moments (CVPR 2024 · similarity 0.88)
• Reinforcing Structured Chain-of-Thought for Video Understanding (CVPR 2026 · similarity 0.88)