Research map / Vision-language and multimodal / Multimodal LLMs

Relation-R1: Progressively Cognitive Chain-of-Thought Guided Reinforcement Learning for Unified Relation Comprehension

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Relation-R1 couples cognitive chain-of-thought SFT (template-based then MLLM-generated CoT) with GRPO using rule-based binary and N-ary relation rewards to unify scene graph generation and grounded situation recognition on Qwen2.5-VL-3B.

Problem

Multimodal LLMs struggle with visual relation understanding, both binary relation detection (scene graph generation) and N-ary relations with multiple semantic roles (grounded situation recognition), often defaulting to language priors.

Approach

A two-stage SFT-then-GRPO framework with cognitive CoT in <think> tags, progressively guided from template-based CoTs to a small number of MLLM-generated CoTs (specific-to-general), plus task-specific rule-based recall rewards for binary and N-ary relations.

Key results

• On PSG caption format, Relation-R1 (3B) reaches 22.33% Recall, 20.07% mRecall, 21.20% Mean, beating ASMv2 (13B) at 12.23 Mean.
• On SWiG open-ended GSR, Relation-R1 reaches 57.26% Verb and 30.18% Grnd-all, improving over OpenSU's 15.70% Grnd-all by about 14.48 points.
• Progressive CoT with joint training reaches 27.19% binary Recall and 73.25% Verb accuracy in the diagnostic study.

Surprising findings

• MLLM-generated CoT underperforms template CoT on N-ary verb-constrained metrics, attributed to plausible but non-GT verbs.
• Reward gains for N-ary relations are attributed to emergent synonymous role and activity generation.

Limitations

• Multi-stage GRPO training is computationally expensive and slow to collect batches.
• GRPO refinement remains partly a black box for interpretability.
• Ablation of SFT-only vs SFT+RL differs across settings, and some comparisons mix caption and standard formats.

Experimental setup

Datasets: PSG SWiG imSitu

Benchmarks: PSG SWiG

Metrics: Recall mRecall Mean Verb Value Value-all Grnd Grnd-all

Baselines: ASMv2 LLaVA-SpaceSGG R1-SGG TextPSG PSGFormer OpenSU GSRFormer CoFormer SituFormer GSRTR

Builds on: DeepSeek-R1 GRPO ASMv2 Visual-RFT

Closest papers by meaning

• Latent Chain-of-Thought for Visual Reasoning (NeurIPS 2025 · similarity 0.86)
• Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards (NeurIPS 2025 · similarity 0.86)
• All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models (CVPR 2026 · similarity 0.86)
• On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs (ICML 2026 · similarity 0.86)
• Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.86)
• ChainMPQ: Interleaved Text-Image Reasoning Chains for Mitigating Relation Hallucinations (ICLR 2026 · similarity 0.86)