Research map / Video / Video-language understanding

CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models

NeurIPS 2026 · Video › Video-language understanding · approach: Event and anomaly reasoning in video · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CollabVR, a training-free closed-loop framework where a VLM (Gemini 2.5 Pro) plans one step at a time, verifies each generated video clip, and folds diagnoses into the next action prompt to improve video generation models on video reasoning tasks.

Problem

Video generation models used for visual reasoning (Chain-of-Frames) suffer from long-horizon drift on multi-step goal-directed tasks and mid-clip execution errors that compound; single-shot generation and best-of-k sampling do not fix these.

Approach

Couple a VLM planner/verifier with a VGM at step-level granularity: progressive planning emits only the next action and adaptively decides step count, while a per-clip verifier accepts or rejects clips and its suggestion is folded into the action prompt for regeneration (plus a failure router for regen/split/fallback).

Key results

• On Gen-ViRe, VBVR-Wan2.2 improves from 0.391 (single inference) to 0.531 with CollabVR; Veo 3.1 improves from 0.481 to 0.550.
• On VBVR-Bench, CollabVR raises VBVR-Wan2.2 overall from 0.671 to 0.757 and Cosmos-Predict2.5 from 0.308 to 0.403.
• In a blind user study (n=40), CollabVR is preferred over Pass@1 in 91.7% and over Pass@4 in 78.9% of decisive votes.

Surprising findings

• M1 (progressive planning) alone hurts Veo 3.1 by 0.035 below single inference, while it helps VBVR-Wan2.2 substantially.
• Forcing maximal decomposition (N=3 for all samples) degrades Veo 3.1 to 0.450, showing adaptive step count matters.

Limitations

• Gains diminish on lower-capability VGMs whose weak instruction-following compounds errors.
• Abstract, analogy, and knowledge categories see small gains since the VGM lacks the underlying capability.
• The VLM verifier is imperfect, letting some failed clips propagate downstream.

Experimental setup

Datasets: Gen-ViRe VBVR-Bench

Benchmarks: Gen-ViRe VBVR-Bench

Metrics: accuracy (rubric score) VGM cost (seconds) human preference

Baselines: Single Inference Pass@2 Pass@4 VideoTPO Pre-planning

Builds on: VideoTPO Chain-of-Thought Pass@k test-time scaling

Closest papers by meaning

• VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video Reasoning? (ICLR 2026 · similarity 0.90)
• Act2See: Emergent Active Visual Perception for Video Reasoning (CVPR 2026 · similarity 0.90)
• OpenCoF: Learning to Reason Through Video Generation (NeurIPS 2026 · similarity 0.89)
• VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding (NeurIPS 2025 · similarity 0.89)
• Look, Remember and Reason: Grounded Reasoning in Videos with Language Models (ICLR 2024 · similarity 0.89)
• Black Swan: Abductive and Defeasible Video Reasoning in Unpredictable Events (CVPR 2025 · similarity 0.89)