Research map / Vision-language and multimodal / Multimodal LLMs

Toward Cognitive Supersensing in Multimodal Large Language Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Cognitive Supersensing, which adds a latent visual imagery prediction head and GFlowNet-based latent-rationale RL to Qwen3-VL-8B, plus a new CogSense-Bench for visual cognition.

Problem

MLLMs struggle with abstract visual cognition (rule induction, mental simulation, visual search) when reasoning is limited to text-space chain-of-thought.

Approach

Augment an MLLM with a Latent Visual Imagery Prediction head that regresses option-image encoder embeddings from backbone hidden states during SFT, then optimize latent rationales with GFlowNet fine-tuning using an answer-evidence plus LVIP grounding reward.

Key results

• CogSense-8B reaches 73.8% average accuracy on CogSense-Bench, versus 40.3% for GPT-5.2 (+33.5) and 35.5% for the base Qwen3-VL-8B.
• LVIP raises SFT average accuracy from 62.3% to 68.0%; adding the GFlowNet RL stage yields 73.8%, beating SFT w/ LVIP + GRPO (70.8%).
• On general benchmarks CogSense-8B stays close to base Qwen3-VL-8B (e.g., AI2D 85.1 vs 85.4, ChartQA 84.7 vs 88.6), and gains +6.2 on EMMA Chemistry and +8.8 on EMMA Math.

Surprising findings

• GPT-o3 scores only 4.7% on Fluid Intelligence, far below its Crystallized Intelligence score of 51.4%.
• Qwen3-VL-8B base scores 35.5% average on CogSense-Bench, while the same model after SFT on the 105K set reaches 62.3% without any LVIP component.

Limitations

• The LVIP target is the embedding of the ground-truth option image, which may limit applicability to tasks without image options.
• Teacher model for rationale generation is unspecified, and the CogSense-Dataset is not confirmed as released.

Experimental setup

Datasets: CogSense-Dataset-105K MaRs-VQA PGM RAVEN Bongard-RWR+ Bongard-HOI Bongard-LOGO KiVA STARE ARC-AGI ARC-AGI-2 CVR

Benchmarks: CogSense-Bench HallusionBench AI2D GQA ScienceQA RealWorldQA ChartQA BLINK MMStar EMMA (Chemistry and Math)

Metrics: accuracy

Baselines: Gemini 2.5 Flash GPT-o3 GPT-5.2 Claude Sonnet 4 Grok 4 Fast Llama-4-Scout-17B Gemma-3-27B Ministral-14B Intern-S1 Qwen3-VL-30B GRPO

Builds on: GFlowNet SubTB GRPO Qwen3-VL

Closest papers by meaning

• Latent Visual Reasoning (ICLR 2026 · similarity 0.91)
• Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens (CVPR 2026 · similarity 0.91)
• VGR: Visual Grounded Reasoning (ICLR 2026 · similarity 0.91)
• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning (ICLR 2026 · similarity 0.91)
• Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning (NeurIPS 2025 · similarity 0.91)
• Monet: Reasoning in Latent Visual Space Beyond Image and Language (CVPR 2026 · similarity 0.91)