Research map / Vision-language and multimodal / Multimodal LLMs

R-C2: Cycle-Consistent Reinforcement Learning Improves Multimodal Reasoning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes R-C2, an RL method that uses cross-modal cycle consistency (answer→query backward inference, then forward inference in the alternate modality) as a label-free binary reward with GRPO, improving Qwen2.5-VL-3B and Qwen3-VL-8B on multimodal reasoning benchmarks.

Problem

Multimodal LLMs give contradictory answers for the same content presented as image versus text, and label-free self-improvement via majority voting amplifies systematic bias ("majority-is-wrong") and is unstable when modalities disagree.

Approach

Replace majority-vote pseudo-labels with a cycle-consistency reward: starting from a candidate answer, the model generates a backward query conditioned on one modality, then answers that query in the alternate modality; reward 1 if the reconstructed answer matches the original. All four modality cycles (T→T, T→I, I→T, I→I) are used.

Key results

• On Qwen2.5-VL-3B, R-C2 raises ScienceQA text/vision accuracy by +7.8/+7.3 points over base and averages +4.8/+2.8 across datasets.
• On Qwen3-VL-8B, R-C2 averages +2.2 text and +1.1 vision accuracy over base, outperforming image+text voting (+1.6/+0.7).
• Cross-modal consistency ratio improves by up to +12.5 points on A-OKVQA and +10.0 on ScienceQA for Qwen2.5-VL-3B.

Surprising findings

• Using the model's own self-generated answers as cycle seeds performs nearly identically to using training-set reference answers on web tasks.
• Training samples with higher cross-modal inconsistency ratios improve both accuracy and consistency, suggesting conflicting samples act as useful hard examples.

Limitations

• Gains on Qwen3-VL-8B are modest (about +1 to +2 points average), and some benchmarks show smaller or negative gains relative to voting.
• Reward depends on answer matching and a binary signal; the approach relies on generated textual views (captions/OCR) for image-only data.
• VWA is repurposed into a custom multiple-choice benchmark with GPT-5-generated questions and human verification, limiting comparability.

Experimental setup

Datasets: ScienceQA ChartQA DocVQA InfoVQA A-OKVQA MathVista VWA (shopping subset, reproposed as MC-QA)

Benchmarks: ScienceQA ChartQA MathVista VWA A-OKVQA DocVQA InfoVQA

Metrics: accuracy (text and vision) cross-modal consistency ratio

Baselines: Base model Voting (Text) Voting (Image+Text) R-Zero

Builds on: GRPO R-Zero majority voting self-consistency

Closest papers by meaning

• On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs (ICML 2026 · similarity 0.90)
• Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning (CVPR 2026 · similarity 0.90)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoning (CVPR 2026 · similarity 0.90)
• Boosting Reasoning in Large Multimodal Models via Activation Replay (CVPR 2026 · similarity 0.90)
• Beyond SFT-to-RL: Pre-alignment via Black-box On-policy Distillation for Multimodal RL (NeurIPS 2026 · similarity 0.89)
• Revisual-R1: Advancing Multimodal Reasoning From Optimized Cold Start to Staged Reinforcement Learning (ICLR 2026 · similarity 0.89)