Research map / Vision-language and multimodal / Multimodal LLMs

Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Robust-U1 equips a unified MLLM (BAGEL) with visual self-recovery of corrupted images via SFT on ImageNet-C, Flow-GRPO RL with SSIM and CLIP rewards, and joint reasoning over corrupted and recovered images, reporting SOTA on R-Bench and anti-degradation VQA benchmarks.

Problem

MLLMs degrade substantially under real-world visual corruptions; existing fixes are either black-box feature alignment or text-only reasoning that cannot restore pixel-level detail.

Approach

Let the MLLM explicitly reconstruct the clean image from the corrupted input (self-recovery), optimize the reconstruction with pixel (SSIM) and semantic (CLIP) rewards via Flow-GRPO, and then reason jointly over corrupted and recovered images.

Key results

• On R-Bench overall, Robust-U1 scores 0.7398 vs. 0.5770 for BAGEL and 0.5017 for Robust-R1.
• On MMMB at 100% corruption, Robust-U1 scores 83.18 vs. 78.48 for BAGEL and 75.35 for Robust-R1.
• Recovery quality on Robust-R1 validation improves PSNR from 14.37 (BAGEL) to 21.49 dB with the full model.

Surprising findings

• Recovery quality (PSNR) gains from SFT alone yield only a small R-Bench gain (+0.02), while RL with either reward gives about +0.13, indicating task alignment matters more than visual fidelity.
• Always-on recovery slightly improves clean-input performance (0.7821 to 0.7865).

Limitations

• Inference is costly: about 55 s latency per sample with 50 denoising steps versus 1.8 s for a standard MLLM.
• Requires paired clean images for the full reward; reference-free variant is weaker.
• Hallucination in recovered images still occurs, with 4.1% harmful flips.

Experimental setup

Datasets: ImageNet-C Robust-R1 training data R-Bench MMMB MMStar RealWorldQA

Benchmarks: R-Bench MMMB MMStar RealWorldQA

Metrics: accuracy GPT-scored mean (VQA/CAP) PSNR SSIM LPIPS

Baselines: Qwen2.5-VL-3B Gemma3-4B InternVL-4B BAGEL TeCoA Robust CLIP Robust LLaVA Robust-R1

Builds on: BAGEL Flow-GRPO Robust-R1 rectified flow SSIM

Closest papers by meaning

• Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding (AAAI 2026 · similarity 0.91)
• Reinforcing Multimodal Reasoning Against Visual Degradation (NeurIPS 2026 · similarity 0.90)
• Machine Vision Therapy: Multimodal Large Language Models Can Enhance Visual Robustness via Denoising In-Context Learning (ICML 2024 · similarity 0.90)
• MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs (ICML 2026 · similarity 0.89)
• Scaling Test-Time Robustness of Vision-Language Models via Self-Critical Inference Framework (CVPR 2026 · similarity 0.89)
• Predictive Regularization Against Visual Representation Degradation in Multimodal Large Language Models (CVPR 2026 · similarity 0.89)