Research map / Vision-language and multimodal / Multimodal LLMs

Think Visually, Reason Textually: Vision-Language Synergy in Abstract Reasoning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that rendering ARC-AGI grids as images helps rule summarization but hurts rule application, and proposes training-free VLSR and MSSC strategies plus a VL-synergy fine-tuning extension that improve ARC-AGI performance.

Problem

Abstract reasoning on ARC-AGI is typically treated as a purely textual task, yet humans use visual abstraction, and naive image rendering of grids degrades LVLM performance.

Approach

Route rule summarization to the visual modality (global 2D pattern perception) and rule application to the textual modality (precise element-wise manipulation), and verify textual outputs by visual consistency checks in a modality-switch self-correction loop.

Key results

• VLSR+MSSC lifts o4-mini from 42.25% to 46.75% on ARC-AGI and Gemini-2.5-Pro from 35.0% to 42.25%, with average gain of about 4.33% over text-only baselines.
• Visual rule application drops accuracy by an average of 15.0% (e.g., Gemini-2.5 from 40.75% to 23.75%), while visual rule summarization gives a 3.2% average gain.
• VL-synergy fine-tuning of Qwen3-8B reaches 13.25% on ARC-AGI, 3.5 points above text-only fine-tuning (9.75%) and above GPT-4o (8.25%).

Surprising findings

• Naively rendering ARC-AGI grids as images degrades performance compared with text-only input during rule application (drop of up to 20.5% for o4-mini).
• Text-only self-correction often stagnates or degrades performance, whereas modality-switch self-correction gives monotonic gains.

Limitations

• Gains are modest in absolute terms (e.g., 4-7 points) and ARC-AGI accuracy remains low for most models.
• Visual rule application is unreliable for element-wise precision, so the approach relies on the text modality for execution.
• Evaluation uses only a subset of Re-ARC and BARC tasks (100 each) and a single sampling setting.

Experimental setup

Datasets: ARC-AGI Re-ARC BARC ARC-Heavy-200k

Benchmarks: ARC-AGI BARC-100 Re-ARC

Metrics: Pass@1 accuracy

Baselines: Text-only reasoning Dynamic Cheatsheet ArcMemo-PS Text-only self-correction

Builds on: Dynamic Cheatsheet ArcMemo test-time training

Closest papers by meaning

• ARC Is a Vision Problem! (CVPR 2026 · similarity 0.91)
• Distinguishing Performance From Competence in Evaluations of Humanlike Abstract Reasoning (NeurIPS 2026 · similarity 0.90)
• The Abstraction Gap in Vision-Language Causal Reasoning (ICML 2026 · similarity 0.90)
• StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning (NeurIPS 2026 · similarity 0.90)
• Synthesizing Visual Concepts as Vision-Language Programs (CVPR 2026 · similarity 0.90)
• Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad? (ICML 2025 · similarity 0.89)