Research map / Vision-language and multimodal / Human and embodied modeling

STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation

NeurIPS 2026 · Vision-language and multimodal › Human and embodied modeling · approach: Vision-language pretraining · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes STARFlow2, a unified multimodal model that vertically interleaves a frozen Qwen2.5-VL-7B with a causal TARFlow normalizing-flow stream via zero-initialized skip connections (Pretzel architecture), enabling text and continuous image generation under one causal mask.

Problem

Existing unified multimodal models combine autoregressive text generation with diffusion image generation (or discrete tokens), creating a structural mismatch that prevents cache-friendly, single-pass interleaved generation while preserving VLM understanding and high-fidelity continuous image genera

Approach

Replace diffusion with autoregressive normalizing flows (TARFlow) and vertically interleave a frozen pretrained VLM with the flow stream through zero-initialized residual skip connections at every position, rather than horizontally separating modalities as in Mixture-of-Transformers.

Key results

• STARFlow2 (10.6B) reaches 0.82 overall on GenEval and 84.94 overall on DPG-Bench, versus 0.51 and 82.02 after Stage 1 only.
• On understanding, STARFlow2 gets MMMU 44.7, MMB 71.5, GQA 55.8, and MME-P 1528.8, below Qwen2.5-VL-Instruct (MMB 83.8, MMMU 50.6) and most native unified baselines on MMB.
• Jointly finetuning the VLM within an MoT-style design drops MME to about 800, motivating the frozen-VLM Pretzel design.

Surprising findings

• The MoT-style horizontal fusion with a frozen VLM gives inferior generation, while finetuning the VLM in MoT degrades understanding (MME ~800).
• Interleaved Stage 3 training raises GenEval from 0.51 to 0.82, a 60.8% relative gain over the generation-only stage.

Limitations

• Multi-stage training pipeline adds complexity and may under-optimize components.
• Resolution and fine-grained fidelity are capped at 256x256 by the frozen FAE encoder.
• Not state-of-the-art on all benchmarks; understanding lags Qwen2.5-VL-Instruct and several native unified baselines.

Experimental setup

Datasets: CC12M JourneyDB Cambrian-7M BLIP3-o-60K CoMM Pico-Banana OmniEdit Zebra-CoT

Benchmarks: MME-P GQA SEED MMBench MMMU AI2D GenEval DPG-Bench

Metrics: MME-P score accuracy GenEval overall DPG-Bench overall

Baselines: BAGEL Janus-Pro Mogao Show-o2 TUNA Emu3 BLIP3-o SD3-Medium FLUX.1 [dev] Qwen-Image Qwen-2.5-VL-Instruct LLaVA-v1.5

Builds on: TARFlow STARFlow Mixture-of-Transformers Feature Auto-Encoder iTARFlow

Closest papers by meaning

• JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation (CVPR 2025 · similarity 0.90)
• LMFusion: Adapting Pretrained Language Models for Multimodal Generation (NeurIPS 2025 · similarity 0.90)
• JetFormer: An autoregressive generative model of raw images and text (ICLR 2025 · similarity 0.89)
• STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis (NeurIPS 2025 · similarity 0.89)
• DuoGen: Towards Autonomous Interleaved Multimodal Generation (CVPR 2026 · similarity 0.89)
• HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation (NeurIPS 2025 · similarity 0.89)