Research map / Vision-language and multimodal / Multimodal LLMs

SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SSL4RL, which turns self-supervised pretext tasks (rotation, jigsaw, contrastive, patch position) into verifiable rewards for GRPO-based RL fine-tuning of VLMs, with extension to graph learning.

Problem

VLMs often under-use visual evidence and rely on textual priors, and RL for VLMs is limited by a lack of scalable, reliable rewards outside domains with programmatic verifiers.

Approach

Reformulate SSL pretext objectives as RL environments: a corruption function yields a context and ground-truth target, and the reward is computed by checking the model's prediction against that target, then optimized with GRPO. No human labels, AI judges, or external verifiers are needed.

Key results

• SSL4RL-Rotation on Qwen2.5-VL-3B improves MMBench average from 72.99% to 80.38%, with Position reaching 80.08%.
• On ImageNet-based Choice-200, SSL4RL-Position raises Qwen2.5-VL-3B from 57.10% to 67.14%.
• On CapArena, the best SSL4RL variant averages 56.45 vs 48.31 for the base model (+8.14); on the graph domain the 3B model gains up to 13.79 points on average.

Surprising findings

• Contrastive SSL, dominant in standard pre-training, degrades several VLM reasoning benchmarks under default augmentations, while position prediction, often seen as too trivial, is among the most effective.
• Combining all four SSL rewards does not yield cumulative gains over the best single reward.

Limitations

• Gains diminish at 7B scale, suggesting predefined SSL tasks become too easy for larger models.
• Naive combination of multiple SSL rewards does not add up; requires more sophisticated weighting.
• Image captioning evaluation relies on GPT-4o as judge on CapArena, not a verifiable reward.

Experimental setup

Datasets: MMBench SEED-Bench ImageNet-1K TAGLAS (graph datasets)

Benchmarks: MMBench SEED-Bench V* RealWorldQA BLINK MME-RealWorld-Lite ImageNet-1K (custom completion/choice) CapArena VLM robustness benchmark Cora PubMed WikiCS

Metrics: accuracy average score CapArena win score against baselines

Baselines: Qwen2.5-VL-3B Qwen2.5-VL-7B Golden-3B (VLM-R1 with golden answers) Jigsaw-R1

Builds on: GRPO PPO VLM-R1 Jigsaw-R1

Closest papers by meaning

• Code as Reward: Empowering Reinforcement Learning with VLMs (ICML 2024 · similarity 0.91)
• VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models (CVPR 2025 · similarity 0.90)
• Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model (NeurIPS 2025 · similarity 0.90)
• PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoning (CVPR 2026 · similarity 0.90)
• Let VLMs Grade Their Own Thoughts: A Self-Quantification Approach to Reasoning-Aware Reward Modeling (CVPR 2026 · similarity 0.90)
• Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning (ICML 2026 · similarity 0.90)