Research map / Vision-language and multimodal / Multimodal LLMs

Children's Intelligence Tests Pose Challenges for MLLMs? KidGym: A 2D Grid-Based Reasoning Benchmark for MLLMs

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces KidGym, a Wechsler-inspired 2D grid-based benchmark with 12 tasks across five MLLM capabilities (execution, perception reasoning, learning, memory, planning), and evaluates closed and open MLLMs against human and random baselines.

Problem

Existing MLLM benchmarks mostly test static, isolated capabilities, so there is limited insight into dynamic, multi-ability performance of multimodal models.

Approach

A procedurally generated, user-extensible 2D grid game suite with randomized layouts and three difficulty levels, whose tasks are designed so that one or two target capabilities dominate, borrowing structure from the Wechsler scales.

Key results

• Gemini-2.5-Pro reaches 0.72 on CO-L1 and o3 reaches 0.83 on FI-L1, while the best PU-L1 score is only 0.30 (GPT-5), close to the 0.25 random baseline.
• Human participants score 1.00 on CO at all three levels, versus 0.72 at best for any model at L1.
• Closed-source models (o3, GPT-5, Gemini-2.5-Pro) reach near-perfect scores on CL, SE and MDE, but success falls sharply on composite tasks such as MMA and MFI.

Surprising findings

• Increasing image resolution from 64 to 96 pixels improved counting accuracy for o3 (0.30 to 0.59) and Claude-3.7-Sonnet (0.54 to 0.79), suggesting reliance on high-resolution cues rather than numerosity.
• In-context learning sometimes performed worse than zero-shot on memory and learning tasks.

Limitations

• Only 12 tasks, which the authors acknowledge is limited in number.
• Human baseline drawn from university students aged 20-25 only.
• Mapping Wechsler constructs onto MLLM capabilities is approximate, and tasks are not purely single-capability (task impurity).

Experimental setup

Benchmarks: KidGym

Metrics: success rate

Baselines: o3 GPT-5 GPT-4o Gemini-2.5-Pro Gemini-2.5-Flash Claude-3.7-Sonnet DeepSeekVL-2 QwenVL-2.5 InternVL-3 Human Random

Builds on: Wechsler Intelligence Scales Gym API Procgen SmartPlay

Closest papers by meaning

• BabyVision: Visual Reasoning Beyond Language (ICML 2026 · similarity 0.89)
• SEED-Bench: Benchmarking Multimodal Large Language Models (CVPR 2024 · similarity 0.88)
• MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.87)
• MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark (ICML 2024 · similarity 0.87)
• Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads (NeurIPS 2024 · similarity 0.87)
• On Path to Multimodal Generalist: General-Level and General-Bench (ICML 2025 · similarity 0.87)