Research map / Vision-language and multimodal / Human and embodied modeling
Show-o2: Improved Native Unified Multimodal Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Show-o2, a native unified multimodal model that couples autoregressive next-token prediction for text with flow matching for image/video latents over a 3D causal VAE space, trained with a two-stage recipe.
Problem
Building a single native unified model that handles multimodal understanding and generation across text, images, and videos without needing a massive text corpus to preserve language ability.
Approach
Unified visual representations built from a 3D causal VAE latent space via a dual-path of semantic layers (distilled from SigLIP) and a projector, fused spatially and temporally; text uses an autoregressive language head while image/video latents use a flow head with flow matching, trained with a two-stage recipe (flow head pre-training, then full-model tuning).
Key results
- On GenEval, the 7B Show-o2 reaches 0.76 overall and the 1.5B model 0.73 using 66M image-text pairs, versus Janus-Pro-7B at 0.80 with 144M pairs.
- On DPG-Bench, the 7B Show-o2 achieves the best overall score of 86.14 among compared models, above Mogao (84.33) and Janus-Pro (84.19).
- On multimodal understanding, the 7B Show-o2 scores 1620.5 on MME-p, 63.1 on GQA, 48.9 on MMMU-val and 79.3 on MMBench, surpassing Janus-Pro-7B on several metrics but trailing Mogao on SEED.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add spatial-temporal fusion (ablation) | Show-o2 pilot with LLaMA-3.2-1B · ImageNet-1K generation pilot | MME-p: 1164.7 -> 1187.8 | Helped |
| add spatial-temporal fusion (ablation) | Show-o2 pilot with LLaMA-3.2-1B · ImageNet-1K generation pilot | GQA: 56.2 -> 57.6 | Helped |
| add spatial-temporal fusion (ablation) | Show-o2 pilot with LLaMA-3.2-1B · ImageNet-1K generation pilot | POPE: 82.6 -> 82.6 | No clear effect |
| add spatial-temporal fusion (ablation) | Show-o2 pilot with LLaMA-3.2-1B · ImageNet-1K generation pilot | FID-5K: 21.8 -> 20.5 | Helped |
| increase CFG guidance from 2.5 to 5.0 (ablation) | Show-o2 1.5B · GenEval | Overall: 0.65 -> 0.71 | Helped |
| increase CFG guidance above 5.0 (ablation) | Show-o2 1.5B · GenEval | Overall: 0.71 -> 0.71 | No clear effect |
| increase CFG guidance from 7.5 to 10 (ablation) | Show-o2 1.5B · DPG-Bench | Overall: 84.8 -> 85.0 | Helped |
| increase inference steps from 25 to 100 at CFG 7.5 (ablation) | Show-o2 1.5B · GenEval | Overall: 0.71 -> 0.73 | Helped |
| add Stage-2 training (ablation) | Show-o2 1.5B · GenEval | Overall: 0.63 -> 0.73 | Helped |
| add Stage-2 training (ablation) | Show-o2 1.5B · DPG-Bench | Overall: 83.28 -> 84.70 | Helped |
| use two-stage training instead of one-stage full-parameter co-training with RefinedWeb (ablation) | Show-o2 1.5B · GSM8K | score: 4.55 ± 0.57 -> 49.43 ± 1.38 | Helped |
| increase image tokens from 729 to 5×729 at inference (ablation) | Show-o2 7B · ChartQA | score: 48.00 -> 66.92 | Helped |
Surprising findings
- One-stage full-parameter co-training with text-only RefinedWeb data collapses language ability (MMLU 28.25 for 1.5B vs 60.20 for Qwen2.5-1.5B-Instruct), so the two-stage recipe is needed.
- On OneIG-Bench text rendering, Show-o2 scores very low (0.002 for 1.5B and 7B), and the paper acknowledges poor text rendering in its limitations.
Limitations
- Poor text rendering in generated images due to a small proportion of rendered-text training images.
- Lacks detail for small objects because of limited image resolution.
- Underperforms LLaVA-OV on ChartQA, attributed to limited chart data in semantic layer distillation.
Experimental setup
Datasets: CC12M COYO LAION-Aesthetic-12M Densefusion-1M LLaVA-OneVision OmniCorpus WebVid Pandas OpenVid-1M VIST CoMM TextAtlas
Benchmarks: MME GQA SEED MMBench MMMU MMStar AI2D POPE Video-MME GenEval DPG-Bench OneIG-Bench
Metrics: accuracy MME-p score GenEval overall score DPG-Bench overall score VBench total score FID-5K
Baselines: Janus-Pro BAGEL Mogao TokenFlow-XL Emu3 Show-o Transfusion ILLUME SEED-X MetaMorph
Builds on: Show-o SigLIP Wan2.1 3D causal VAE DiT adaLN-Zero Qwen2.5 Instruct
Closest papers by meaning
- Show-o: One Single Transformer to Unify Multimodal Understanding and Generation (ICLR 2025 · similarity 0.90)
- Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.88)
- MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer (ICLR 2026 · similarity 0.88)
- STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation (NeurIPS 2026 · similarity 0.88)
- FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities (NeurIPS 2025 · similarity 0.88)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.