Research map / Vision-language and multimodal / Multimodal LLMs

Empowering Small VLMs to Think with Dynamic Memorization and Exploration

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DyME, which per-sample switches between SFT memorization and GRPO exploration for small VLMs, plus a visual checker/refiner that builds image-grounded CoT supervision.

Problem

Small VLMs (under ~1B parameters) cannot acquire thinking abilities via SFT on CoT data (pseudo thinking traces) or RLVR (advantage collapse from poor instruction following); static SFT+RL blends are brittle.

Approach

Per training step, use GRPO when at least one sampled response is correct and fall back to SFT on ground-truth otherwise, combined with a visual checker and refiner (LLM-based) that score exploration traces and rewrite ground-truth into image-grounded reasoning.

Key results

• DyME lifts SmolVLM (0.5B) average from 49.9 to 55.6 (+5.7) and LLaVA-OV-S from 50.7 to 55.4 (+4.7) across medical, chart, and geometry.
• CoT SFT drops SmolVLM to 44.1 and GRPO to 44.0, while two-stage training gives 45.4.
• Pure DyME reaches 64.9 on ChartQA with medium-quality data vs 57.8 for SFT on the same data.

Surprising findings

• Two-stage SFT->RL often falls below the untrained baseline on SVLMs, e.g. LLaVA-OV-S chart score drops to 52.9.
• SVLMs converge within one SFT epoch; performance after 1 epoch matches or exceeds 10 epochs.

Limitations

• Visual supervision relies on extractable visual facts, so it may not apply to abstract domains such as memes or dense crowds.
• Gains are reported on specialized tasks with a few thousand samples; generality beyond these domains is limited to supplementary experiments.
• Visual facts and refinement depend on external LLMs (Qwen2.5-14B/GPT-4o) and domain tools.

Experimental setup

Datasets: SLAKE ChartQA Geo170K A-OKVQA GSM8K

Benchmarks: SLAKE ChartQA MathVerse GSM8K A-OKVQA

Metrics: accuracy recall relaxed correctness average accuracy

Baselines: CoT SFT GRPO Two-stage SFT->RL CHORD LUFFY LLaVA-1.6 MoVA LLaVA-OV-L InternVL2-L Cambrian-1 LLaVA-Med

Builds on: GRPO DeepSeek-R1 CHORD LUFFY R1-V

Closest papers by meaning

• See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection (CVPR 2026 · similarity 0.91)
• Code as Reward: Empowering Reinforcement Learning with VLMs (ICML 2024 · similarity 0.89)
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
• Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model (NeurIPS 2025 · similarity 0.89)
• VLMs-Guided Representation Distillation for Efficient Vision-Based Reinforcement Learning (CVPR 2025 · similarity 0.89)
• SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning (ICML 2026 · similarity 0.89)