Research map / Vision-language and multimodal / Multimodal LLMs
Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Post-trains the unified VILA-U model with a small interleaved warm-up and a GRPO variant using hybrid text/visual/format rewards plus process-level rewards to improve multimodal interleaved generation on MMIE and InterleavedBench.
Problem
Unified vision-language models struggle to produce coherent multimodal interleaved text-image outputs, and large-scale high-quality interleaved training data is scarce.
Approach
Activates interleaved generation with a small hybrid warm-up dataset, then extends GRPO to treat text and image tokens as one decoding trajectory, optimized with hybrid rewards (textual relevance, image quality via ImageReward, format) and step-wise process rewards.
Key results
- Full method reaches 59.50 on MMIE, versus 55.22 for Anole (best baseline in Table 1) and 57.27 for the QLoRA variant.
- On InterleavedBench, the method scores 3.13 average, a gain of 1.29 over GILL (1.84).
- Warm-up alone gives 53.31 on MMIE; adding GRPO raises it to 59.50, with the reward ablation showing visual reward adds 3.01 points on MMIE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply the full warm-up and GRPO training strategy (ablation) | VILA-U 7B · MMIE | score: 53.31 -> 59.50 | Helped |
| apply the full warm-up and GRPO training strategy (ablation) | VILA-U 7B · InterleavedBench | score: 1.97 -> 3.13 | Helped |
| add warm-up training to the untrained baseline (ablation) | VILA-U 7B · MMIE | score | Helped |
| add textual reward to format reward (ablation) | GRPO-trained VILA-U 7B · MMIE | score: 53.56 -> 54.62 | Helped |
| add textual reward to format reward (ablation) | GRPO-trained VILA-U 7B · InterleavedBench | score: 2.05 -> 2.30 | Helped |
| add visual reward to format and textual rewards (ablation) | GRPO-trained VILA-U 7B · MMIE | score: 54.62 -> 57.83 | Helped |
| add visual reward to format and textual rewards (ablation) | GRPO-trained VILA-U 7B · InterleavedBench | score: 2.30 -> 2.79 | Helped |
| add process-level reward to hybrid rewards (ablation) | GRPO-trained VILA-U 7B · MMIE | score: 57.83 -> 59.50 | Helped |
| add process-level reward to hybrid rewards (ablation) | GRPO-trained VILA-U 7B · InterleavedBench | score: 2.79 -> 3.13 | Helped |
| add KL penalty (ablation) | GRPO-trained VILA-U 7B with G=2 and ImageReward · MMIE | score: 53.04 -> 55.14 | Helped |
| increase sampled generations from 2 to 4 (ablation) | GRPO-trained VILA-U 7B with KL and ImageReward · MMIE | score: 55.14 -> 59.50 | Helped |
| increase sampled generations from 2 to 4 (ablation) | GRPO-trained VILA-U 7B with KL and ImageReward · InterleavedBench | score: 2.27 -> 3.13 | Helped |
Surprising findings
- The authors report that GRPO brings no notable gains on general understanding and text-to-image benchmarks, which they attribute to the base model's limits.
- CLIP-score as the visual reward underperforms ImageReward (56.94 vs 59.50 on MMIE).
Limitations
- Gains do not extend to general understanding or text-to-image generation tasks.
- GRPO is resource-intensive in multimodal settings because each rollout generates both text and images; only G=2 and G=4 were tested.
- Failure cases include hallucinated visual content and style inconsistency with input images.
Experimental setup
Datasets: ActivityNet GenHowTo OpenStory++ EMOVA JourneyDB
Benchmarks: MMIE InterleavedBench MME-P MMvet SEEDBench-img POPE GenEval
Metrics: MMIE score InterleavedBench score (text quality, perceptual quality, im GenEval score
Baselines: MiniGPT-5 EMU-2 GILL Anole VILA-U SEED-X TokenFlow-XL Janus-Pro Chameleon ARMOR
Builds on: GRPO VILA-U ImageReward PPO DPO
Closest papers by meaning
- DuoGen: Towards Autonomous Interleaved Multimodal Generation (CVPR 2026 · similarity 0.91)
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.90)
- Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models (CVPR 2026 · similarity 0.90)
- CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.90)
- ParaUni: Enhance Generation in Unified Multimodal Model with Reinforcement-driven Hierarchical Parallel Information Interaction (CVPR 2026 · similarity 0.89)
- Co-Reinforcement Learning for Unified Multimodal Understanding and Generation (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.