Research map / Vision-language and multimodal / Multimodal LLMs

BabyVision: Visual Reasoning Beyond Language

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces BabyVision, a 388-question benchmark of pre-linguistic visual skills evaluated on MLLMs versus human age groups, plus BabyVision-Gen for visual-output reasoning and a small RLVR study on Qwen3-VL-8B.

Problem

Current multimodal LLMs excel at knowledge-heavy benchmarks but fail on basic, language-free visual tasks that young children solve easily; existing benchmarks rarely measure these beyond-language visual abilities systematically.

Approach

A benchmark of 388 vision-centric questions across 22 subtypes in four categories, calibrated against human testers aged 3 to adults, with a generative counterpart (BabyVision-Gen) that scores visual annotations produced by image generators instead of verbal answers.

Key results

• Gemini3-Pro-Preview scores 49.7% on BabyVision versus 94.1% for adult humans and about 20 points below 6-year-olds.
• On BabyVision-Gen, NanoBanana-Pro reaches 18.3% overall, the best of three generation models evaluated.
• GRPO on Qwen3-VL-8B-Thinking with 1,400 training examples raises BabyVision accuracy from 13.1% to 17.9% (+4.8).

Surprising findings

• Visual tracking tasks such as Maze and Connect the Lines reach 0% for all generation models on BabyVision-Gen.
• RLVR gains are absent or negative on visual tracking subtypes, which the authors attribute to limited verbalizability.

Limitations

• Human testers for the age comparison come from a single school population and only 20 items (BabyVision-Mini) were used for children.
• Auto-judge agreement on BabyVision-Gen was measured only on NanoBanana-Pro outputs; the reported confusion-matrix accuracy (25%) conflicts with the 96.1% agreement figure in the text.
• Benchmark is small (388 questions), so subtype results rest on few items.

Experimental setup

Datasets: BabyVision BabyVision RLVR training set (1,400 examples)

Benchmarks: BabyVision BabyVision-Gen

Metrics: accuracy (Avg@3) standard deviation across runs agreement with human judgments F1

Baselines: Gemini3-Pro-Preview GPT-5.2 Doubao-1.8 Qwen3-VL-Plus Claude-4.5-Opus Grok-4 Qwen3VL-235B-Thinking InternVL3.5-241B GLM4.6V MimoVL-7B-RL Step3 KimiVL-A3B

Builds on: GRPO EasyR1 Qwen3-VL

Closest papers by meaning

• BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models (CVPR 2026 · similarity 0.91)
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.89)
• Children's Intelligence Tests Pose Challenges for MLLMs? KidGym: A 2D Grid-Based Reasoning Benchmark for MLLMs (ICLR 2026 · similarity 0.89)
• KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models (ICLR 2025 · similarity 0.89)
• Caption This, Reason That: VLMs Caught in the Middle (NeurIPS 2025 · similarity 0.89)
• VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models (ICLR 2026 · similarity 0.88)