Research map / Vision-language and multimodal / Multimodal LLMs

Perceptual Flow Network for Visually Grounded Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PFlowNet, which models visually grounded reasoning as sampling structured perceptual flows (planning state plus RoI-caption chains) from a self-parameterized variational distribution, trained via SFT and a variational RFT with a multi-dimensional reward and vicinal geometric shaping.

Problem

LVLMs hallucinate and show language bias in visually grounded reasoning; existing methods distill geometric priors from visual experts (e.g., GroundingDINO) as RLVR rewards, but these expert priors are biased toward localization precision rather than reasoning utility.

Approach

Replace rigid alignment to expert boxes with a self-parameterized perceptual-flow distribution trained with Sub-Trajectory Balance; the reward combines a contrastive caption-grounding term with answer likelihood, and a vicinal geometric shaping weight penalizes RoIs outside an IoU-vicinity of expert annotations.

Key results

• PFlowNet improves Qwen3-VL-8B by 10.4% on TreeBench (55.3 vs 44.9) and 18.4% on MME-RealWorld-Lite.
• On V* Bench PFlowNet reaches 90.6 overall vs 77.5 for the Qwen3-VL-8B base, matching or exceeding Qwen3-VL-32B (87.4).
• Ablation: removing the perceptual flow at inference drops TreeBench accuracy from 54.5 to 49.2 and V* from 89.4 to 83.8.

Surprising findings

• The most precise expert annotation is not the most helpful evidence for reasoning; expanded RoIs yield better accuracy when cropped evidence is fed to the model.
• Incorporating external fine-grained visual features at inference yields only marginal gains compared with using the textual perceptual flow.

Limitations

• Theoretical bound relies on idealized assumptions and regularity conditions that may not hold in practice.
• Requires tuning of vicinal radius epsilon and shaping intensity lambda, which may vary across models and domains.
• Fixed structured reasoning format lacks adaptive perception and adds overhead on simple questions.

Experimental setup

Datasets: V* expert-annotated RoI datasets (cross-expert annotated)

Benchmarks: TreeBench MME-RealWorld-Lite V* Bench HR-Bench 4K HR-Bench 8K ScreenSpot ScreenSpot-Pro MMBench AI2D ChartQA

Metrics: accuracy mIoU pass@k latency

Baselines: TreeVGR Pixel-Reasoner DeepEyes DeepEyesV2 Thyme Qwen3-VL-8B ViGoRL

Builds on: Sub-Trajectory Balance GRPO-style RLVR TreeVGR Vicinal Risk Minimization

Closest papers by meaning

• GFlowVLM: Enhancing Multi-step Reasoning in Vision-Language Models with Generative Flow Networks (CVPR 2025 · similarity 0.87)
• Artemis: Structured Visual Reasoning for Perception Policy Learning (ICML 2026 · similarity 0.86)
• From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.86)
• Improving Vision-language Models with Perception-centric Process Reward Models (CVPR 2026 · similarity 0.86)
• Grounded Reinforcement Learning for Visual Reasoning (NeurIPS 2025 · similarity 0.85)
• CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solving (ICLR 2026 · similarity 0.85)