Research map / Vision-language and multimodal / Multimodal LLMs

UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation

NeurIPS 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds UniGen, a 1.5B unified MLLM trained on open-source data through PT, SFT, DPO and a lightweight CoT-verification post-training, which enables best-of-N test-time self-verification for text-to-image generation.

Problem

Building unified multimodal LLMs that both understand and generate images, with little guidance on how training stages and test-time interaction between understanding and generation should be combined, and reliance on internal datasets in prior work.

Approach

Study the full training life cycle from a data-centric view (multi-stage pretraining, SFT, DPO with GenEval/VQA-based preference pairs) and use the model itself as a chain-of-thought verifier that scores each generated image by answering atomic visual questions, selecting the best of N candidates at test time.

Key results

• UniGen reaches 0.78 overall on GenEval and 85.19 on DPG-Bench, versus 0.73 for Janus-Pro on GenEval and 82.63 on DPG-Bench.
• CoT-V test-time verification raises GenEval from 0.73 (DPO) to 0.78 and DPG-Bench from 84.89 to 85.19; outcome verification gives no gain.
• On understanding benchmarks, UniGen (1.5B) gets 56.7 on RealWorld-QA and 44.6 on MathVista, above Janus-Pro-1.5B (51.1 and 36.8).

Measured effects reported in the paper

Surprising findings

• Outcome verification (binary yes/no without reasoning) gives no improvement over the base model, while CoT-style step-by-step verification gives a clear boost.
• Using ImageNet with fine-grained captions instead of class names in PT-1 improves understanding performance, and removing understanding data from pre-training severely hurts generation as well.

Limitations

• Only a 1.5B model is instantiated and evaluated, limited by compute.
• Generation is restricted to 256x256 resolution.
• CoT verification is limited on complex free-form prompts due to noisy pseudo-labels from Qwen2.5-VL.

Experimental setup

Datasets: ImageNet CC-3M CC-12M SAM-11M (SA-1B) RefinedWeb SlowFast-LLaVA-1.5 JourneyDB text-2-image-2M PARM prompts T2I-CompBench DPG-Bench prompts

Benchmarks: GQA RealWorld-QA SEED-Bench AI2D MMMU MathVista POPE ScienceQA GenEval DPG-Bench

Metrics: accuracy GenEval overall score DPG-Bench overall score

Baselines: Show-o Janus Janus-Pro Vila-U MMAR UniToken ILLUME VARGPT-v1.1 TokenFlow-XL LLaVA-OV MM1.5 LLaVA 1.5

Builds on: Show-o MaskGIT LLaVA DPO Janus-Pro

Closest papers by meaning

• SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding (CVPR 2025 · similarity 0.90)
• MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models (ICLR 2026 · similarity 0.89)
• On Fairness of Unified Multimodal Large Language Model for Image Generation (NeurIPS 2025 · similarity 0.89)
• UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark (CVPR 2026 · similarity 0.89)
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer (ICLR 2026 · similarity 0.89)
• LMFusion: Adapting Pretrained Language Models for Multimodal Generation (NeurIPS 2025 · similarity 0.89)