Research map / Vision-language and multimodal / Multimodal LLMs

Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matching

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UniDFlow, a unified discrete flow-matching model for multimodal understanding, text-to-image generation, and editing, built on a frozen pretrained VLM with LoRA adapters and a reference-based DPO-style preference alignment (mRef-DPO).

Problem

Existing unified multimodal models mix mismatched objectives (AR cross-entropy plus diffusion regression), entangle understanding and generation parameters, and align generation/editing with scalar rewards or reflection that lack relative preference under identical conditioning.

Approach

Repurpose a frozen pretrained VLM as a discrete flow-matching denoiser over text and visual tokens, with separate understanding and generation LoRA adapters combined by a learned router, time conditioning via zero-initialized modulation of RMSNorm scales, and a reference-anchored DPO objective applied to text, visual, and reflection outputs.

Key results

• UniDFlow-4B reaches GenEval 0.95 and DPGBench 91.19, above EMMA (0.93 / 85.63) and Qwen-Image (0.91 / 88.32).
• On ImgEdit overall, UniDFlow scores 4.24 versus EMMA 4.01 and MammothModa2 4.06.
• UniDFlow-14B reaches OCRBench v2 average 63.8 versus 56.1 for MammothModa2.

Surprising findings

• Sharing a single LoRA for understanding and generation fails badly (EvalVLM 79.92 vs 82.85), and in the authors' words 'fails entirely' on the ablation.
• Vanilla DPO and uni-GRPO each underperform mRef-DPO, with DPO lowering EvalVLM to 80.12 from 82.85.

Limitations

• Stage III relies on preference data generated with proprietary multimodal LLMs, and alignment quality depends on the diversity of that supervision.
• Training data is large (3.6T tokens total), so the parameter-efficiency claim rests on LoRA rather than total compute.
• Several baseline numbers appear inconsistent across tables (e.g., DPGBench overall for UniDFlow differs between 91.19 and 93.42 in the appendix), and the paper gives no code or weight release.

Experimental setup

Datasets: MMInstruct Text-to-Image-4M LAION-5B JourneyDB text-to-image-2M OpenGPT-4o-Image AnyEdit Pico-Banana-400K PIE-Bench

Benchmarks: EvalVLMBench GenEval DPGBench ImgEdit Emu-Edit GEdit-Bench-EN OCRBench v2 PIE-Bench

Metrics: accuracy overall score CLIP-I CLIP-Out DINO SC PQ PSNR SSIM

Baselines: BAGEL EMMA MammothModa2 MUDDIT OmniGenV2 Janus-Pro-7B Qwen-Image Qwen2.5-VL-3B FLUX.1 Kontext-Pro UniWorld-V1 DPO uni-GRPO

Builds on: Discrete Flow Matching Direct Preference Optimization Diffusion-DPO PyraTok

Closest papers by meaning

• Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners (ICML 2026 · similarity 0.89)
• HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation (NeurIPS 2025 · similarity 0.87)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.87)
• UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing (ICLR 2026 · similarity 0.87)
• UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark (CVPR 2026 · similarity 0.87)
• TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.86)