Research map / Vision-language and multimodal / Multimodal LLMs

Reinforcing Multimodal Reasoning Against Visual Degradation

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ROMA, a critic-free RL fine-tuning method for MLLMs that adds a correctness-gated, token-level KL invariance penalty over worst-case degraded views plus an auxiliary clipped policy-gradient loss anchored to clean advantages, improving robustness to visual corruptions on Qwen3-VL 4B/8B.

Problem

RL-fine-tuned multimodal reasoning models are brittle to real-world visual degradations (blur, compression, low-resolution), and existing invariance/augmentation robustness methods from vision and deep RL do not transfer to critic-free GRPO-style training without causing reward poisoning.

Approach

Dual-forward-pass training: rollouts are sampled only on clean images; degraded views are scored via teacher forcing on the same clean trajectories. Adds a worst-case (max over K augmentations) token-level KL penalty, applied only to correct trajectories, plus an auxiliary PG loss using clean-image advantages on a sampled degraded view.

Key results

• On Qwen3-VL-8B, ROMA reaches 61.6% on seen degradations vs 59.2% for GRPO (+2.4) while matching clean accuracy (68.7% vs 68.9%).
• On unseen degradations at 8B, ROMA reaches 56.3% vs 54.0% for GRPO (+2.3).
• Replacing the worst-case penalty with a mean penalty drops seen accuracy from 61.6 to 60.0 at 8B; removing the auxiliary PG loss drops it to 60.5.

Surprising findings

• Unconditional invariance penalty (without correctness gating) reduces seen accuracy to 59.4%, below the GRPO baseline of 59.2% only marginally above, indicating the gating matters.

Limitations

• Gains are modest (about +2.3 to +2.4 points) and evaluated on a single run without reported variance.
• Only four seen and five unseen synthetic degradation types are tested; real-world degradations are not directly measured.
• Hyperparameters (alpha, beta, K) were tuned on the same benchmarks reported in the main results.

Experimental setup

Datasets: MMRL30k

Benchmarks: MathVista WeMath ChartQA LogicVista MMStar VisualPuzzles RealWorldQA

Metrics: accuracy

Baselines: GRPO Qwen3-VL Instruct (base) NoisyRollout-7B PAPO-7B Vision-R1-7B VL-Rethinker-7B OpenVLThinker-7B

Builds on: GRPO DrAC EasyR1

Closest papers by meaning

• On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs (ICML 2026 · similarity 0.92)
• Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding (AAAI 2026 · similarity 0.92)
• Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.91)
• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.91)
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.90)
• Activating Visual Context and Commonsense Reasoning Through Masked Prediction in VLMs (AAAI 2026 · similarity 0.90)