Research map / Vision-language and multimodal / Multimodal LLMs

Act2See: Emergent Active Visual Perception for Video Reasoning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Act2See fine-tunes Qwen3-VL-8B-Thinking with SFT on human-verified interleaved video-text CoTs in which the model calls frame retrieval or generation tools, enabling active visual perception during video reasoning.

Problem

Video VLMs reason over static initial frames, so they cannot gather new visual evidence mid-reasoning or imagine counterfactual/hypothetical scenes, and existing frame-augmented CoTs are often built from noisy VLM-generated traces.

Approach

Generate interleaved video-text CoTs with a frontier VLM that emits retrieve/generate tool calls (retrieval via TFVTG, generation via Stable Diffusion 3.5 conditioned on a retrieved frame), filter them by similarity to human-annotated CoTs from MINERVA, CausalVQA and Social Genome, and SFT a smaller VLM so it learns to decide when to retrieve or synthesize frames at inference.

Key results

• Act2See reaches 74.2 on Video-MME vs 71.8 for its base Qwen3-VL-8B-Thinking, and 46.8 on VideoEspresso vs GPT-4o's 26.4.
• On ViTIB Act2See scores 63.3, versus 60.2 for Qwen3-VL-8B-Thinking and 56.8 for InternVL2.5-8B.
• Act2See scores 47.1 on VCR-Bench, above Qwen3-VL-8B-Thinking (38.2) and ReWatch-R1 on Qwen2.5-VL-7B (39.6 vs 42.2 for Act2See on the same base).

Surprising findings

• Feeding ground-truth CoTs into the Gemini prompt sharply reduced tool-call rate (5.26% vs 45.21% of CoTs with frames) and lowered downstream Video-MME accuracy (72.2 vs 73.7).
• SFT on VLM-generated CoTs (VideoEspresso, Video-R1) produced results that could be worse than the base model, while human-annotated sources helped.

Limitations

• Generated frames sometimes do not faithfully follow the generation prompt (e.g., coffee beans poured into the mug instead of the machine).
• Training data is small (3,373 samples) and depends on a frontier VLM and human-annotated benchmarks for quality filtering.
• Sample-level manual check covered only 100 CoTs.

Experimental setup

Datasets: MINERVA CausalVQA Social Genome Video-MME VideoEspresso EgoNormia VCR-Bench ViTIB

Benchmarks: Video-MME VideoEspresso EgoNormia VCR-Bench ViTIB

Metrics: accuracy

Baselines: GPT-4o Gemini 2.5 Pro Qwen2.5-VL-7B InternVL2.5-8B Video-LLaMA3-7B Qwen3-VL-8B-Thinking Video-R1 Chain-of-Shot FrameMind ViTCoT Chain-of-Frames ReWatch-R1

Builds on: Chain-of-Thought prompting ViTCoT Video-R1 TFVTG

Closest papers by meaning

• BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception (NeurIPS 2025 · similarity 0.91)
• When Thinking Drifts: Evidential Grounding for Robust Video Reasoning (NeurIPS 2025 · similarity 0.90)
• From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.90)
• CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models (NeurIPS 2026 · similarity 0.90)
• VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding (NeurIPS 2025 · similarity 0.90)
• V-ABS: Action-Observer Driven Beam Search for Dynamic Visual Reasoning (ICML 2026 · similarity 0.90)