Research map / Vision-language and multimodal / Multimodal LLMs

Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Think-as-You-See (TaYS), a streaming chain-of-thought SFT framework for LVLMs that reasons while video frames arrive, using streaming attention masks, decoupled positional encoding, and parallel dual KV caches, evaluated on an extended VideoEspresso with Qwen2.5-VL-3B/7B.

Problem

Existing LVLM video reasoning is batch-based: it waits for the full video before reasoning, adding latency and temporal drift. Real-time streaming video needs reasoning that stays synchronized with incoming frames.

Approach

Shift LVLM video CoT to a streaming paradigm: train on frame-aligned reasoning trajectories with a causal sliding-window attention mask, modality-decoupled position indices (pos(v_s)=s, pos(r_t)=t), and separate video and text KV caches merged at decode time so visual encoding and reasoning run concurrently.

Key results

• TaYS improves Qwen2.5-VL-7B accuracy to 36.86 vs 30.38 for Batch SFT on the extended VideoEspresso benchmark, while Interleaved SFT reaches 34.98.
• TaYS reaches a 43.7% normalized GPT-5 win rate vs 31.4% for Batch and 21.7% for Interleaved.
• Batch TTFT is about 10.6s while TaYS decoder-level TTFT is near zero; mean reasoning-event deviation drops from 1.52s to 0.69s.

Surprising findings

• The Interleaved SFT model achieves slightly higher objective accuracy than TaYS in the main comparison (e.g., 33.96 vs 33.45 for 3B), which the authors attribute to both streaming paradigms capturing temporal dependencies.
• Batch with thinking performs about the same as or worse than batch without thinking for the 3B model (28.16 vs 27.99 is a small gain, but reasoning hurts several categories).

Limitations

• Objective metric is a semantic-similarity threshold with multiple-choice distractors, which may not capture reasoning coherence; subjective evaluation depends on GPT-5 judging.
• Interleaved SFT matches or beats TaYS on some objective accuracy measures, so the accuracy gain is not uniform.
• Evaluation uses an extended, GPT-4o-generated VideoEspresso-derived benchmark, and the TTFT measurement is at decoder level and under warm-start conditions.

Experimental setup

Datasets: VideoEspresso

Benchmarks: VideoEspresso (extended)

Metrics: accuracy win rate TTFT overall delay reasoning-event deviation

Baselines: Batch w/o thinking Batch with thinking Batch SFT Interleaved SFT

Builds on: Qwen2.5-VL VideoEspresso RoPE

Closest papers by meaning

• StreamingThinker: Large Language Models Can Think While Reading (ICLR 2026 · similarity 0.92)
• SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding (ICLR 2025 · similarity 0.90)
• VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video Reasoning? (ICLR 2026 · similarity 0.90)
• When Thinking Drifts: Evidential Grounding for Robust Video Reasoning (NeurIPS 2025 · similarity 0.90)
• Act2See: Emergent Active Visual Perception for Video Reasoning (CVPR 2026 · similarity 0.90)
• Look, Remember and Reason: Grounded Reasoning in Videos with Language Models (ICLR 2024 · similarity 0.89)