Research map / Vision-language and multimodal / Multimodal LLMs

ParaUni: Enhance Generation in Unified Multimodal Model with Reinforcement-driven Hierarchical Parallel Information Interaction

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ParaUni, which feeds learnable-query features from all VLM layers in parallel through a Layer Integration Module into a diffusion model, and uses a Layer-wise Dynamic Adjustment Mechanism during Flow-GRPO RL to perturb layers according to reward and gradient-norm signals.

Problem

Existing unified multimodal models condition diffusion generators on only the last VLM layer (insufficient detail) or couple VLM and diffusion tightly (inflexible, costly), and multi-reward RL does not account for differing layer roles.

Approach

Aggregate learnable-query features from every VLM layer in parallel via a shared transformer plus LayerNorm (LIM) as the diffusion condition, and during RL inject Gaussian noise into specific layers chosen by which reward is declining and by gradient-norm spikes (LDAM).

Key results

• ParaUni reaches GenEval overall 0.87 and DPG-Bench 83.45, above OpenUni (0.86 / 83.08) and BAGEL (0.82 GenEval).
• Plugged into Janus-Pro (1B), ParaUni raises GenEval from 0.73 to 0.80 and DPG-Bench from 82.63 to 83.65.
• Plugged into BLIP3-o (4B), ParaUni raises GenEval from 0.81 to 0.84 and DPG-Bench from 79.36 to 81.97.

Surprising findings

• Ablation shows removing the LayerNorm in LIM drops colors score to 0.61 and overall GenEval to 0.73, the largest drop among ablations.

Limitations

• Relies on a frozen VLM, so understanding ability is inherited from InternVL3 and not improved by the method.
• Layer-wise perturbation scale and RL configuration are deferred to supplementary material, limiting reproducibility from the main text.
• Layer-selection ranges and reward-to-layer mapping are derived from empirical observations on one base model.

Experimental setup

Datasets: text-to-image-2M LAION-Aesthetic-6M Megalith-10M RedCaps-5M BLIP3-o-60k

Benchmarks: GenEval DPG-Bench

Metrics: GenEval overall score DPG-Bench overall score CLIP score Aesthetic score Pickscore

Baselines: OpenUni BAGEL BLIP3-o MetaQuery Janus-Pro Show-o SD3-Medium Emu3-Gen

Builds on: MetaQuery OpenUni Flow-GRPO Diffusion-DPO

Closest papers by meaning

• Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization (NeurIPS 2025 · similarity 0.89)
• Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models (CVPR 2026 · similarity 0.89)
• DuoGen: Towards Autonomous Interleaved Multimodal Generation (CVPR 2026 · similarity 0.88)
• Co-Reinforcement Learning for Unified Multimodal Understanding and Generation (NeurIPS 2025 · similarity 0.88)
• Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models (NeurIPS 2026 · similarity 0.87)