Research map / Vision-language and multimodal / Multimodal LLMs

TTRV: Test-Time Reinforcement Learning for Vision Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TTRV, a test-time GRPO framework that adapts VLMs on unlabeled test samples using a frequency-based self-consistency reward plus an entropy-based diversity reward.

Problem

Vision-language models are static after training and rely on labeled data and curated splits for RL-based post-training; the paper asks whether RL can instead be run directly on unlabeled test data at inference time.

Approach

Extends GRPO to test time with rewards computed from the empirical frequency of the model's own sampled answers per test sample (soft frequency weighting rather than best-of-N majority voting), combined with a negative-entropy term that controls output diversity.

Key results

• On image classification, TTRV on InternVL3-8B reaches 95.71 mean accuracy versus 66.74 base (+28.97), with 99.31% on ImageNet, and TTRV on InternVL-8B is reported to surpass GPT-4o by 2.3% average over 8 recognition benchmarks.
• On VQA, TTRV on InternVL2.5-4B improves mean accuracy from 66.37 to 69.40 across 8 benchmarks; AI2D gains +9.54.
• Single-sample TTRV on InternVL2.5-4B yields +5.47 on ImageNet-R and +4.61 on ImageNet-A.

Surprising findings

• Random rewards do not help InternVL models, in contrast to reported spurious-reward gains on Qwen models.
• TTRV on InternVL2.5-4B decreases accuracy on Resisc45 (-10.14), attributed to low base-model quality producing poor rollouts.

Limitations

• No theoretical explanation for why test-time RL enhances task ability rather than adapting to data distribution.
• Performance can degrade when the base model is weak (e.g., Resisc45 with InternVL2.5-4B).
• Adaptation latency grows substantially with number of adaptation samples.

Experimental setup

Datasets: ImageNet ImageNet-V2 ImageNet-R ImageNet-S ImageNet-A Food101 DTD Resisc45 MathVerse MathVista SEED MME

Benchmarks: ImageNet ImageNet-V2 ImageNet-R ImageNet-S ImageNet-A Food101 DTD Resisc45 MathVerse MathVista SEED MME

Metrics: top-1 accuracy mean accuracy 1 - symmetric mean percentage error

Baselines: GPT-4o CLIP MetaCLIP EVA-CLIP SigLIP LLaMA-3.2-11B LLaVA-1.5-7B Phi-3.5-vision TTRL majority voting TENT-style entropy minimization random rewards

Builds on: GRPO TTRL TENT TPT DeepSeekMath GRPO

Closest papers by meaning

• AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition (CVPR 2026 · similarity 0.91)
• Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models (ICLR 2024 · similarity 0.91)
• Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.91)
• The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.90)
• VisPlay: Self-Evolving Vision-Language Models (CVPR 2026 · similarity 0.89)
• PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoning (CVPR 2026 · similarity 0.89)