Research map / Vision-language and multimodal / Multimodal LLMs

UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generation

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UniFlow, a unified tokenizer that adapts any pretrained visual encoder via layer-wise adaptive self-distillation and a lightweight patch-wise pixel flow decoder, aiming to balance understanding and reconstruction/generation.

Problem

Existing unified visual tokenizers trade off semantic understanding against high-fidelity pixel reconstruction, because high-level semantic abstraction conflicts with low-level pixel detail.

Approach

Distill a pretrained VFM encoder into a unified encoder with layer-wise adaptive weights (deeper layers weighted more, boosted by alignment penalty), and decode pixels via a patch-wise rectified-flow decoder conditioned on semantic latents, with a global transformer block to avoid grid artifacts.

Key results

• UniFlow(InternViT) reaches 0.26 rFID on ImageNet-1K, beating UniTok (0.41) and SD-VAE, with 33.23 PSNR at 448 resolution.
• UniFlow-XL (7B, Qwen2.5-7B) averages 79.09 across understanding benchmarks, reported as surpassing the 14B TokenFlow-XL by 6.05% on average.
• A 0.6B UniFlow-based MMDiT achieves 0.65 GenEval and 84.76 DPG-Bench at 256 resolution, above SANA-0.6B and TokenFlow-7B.

Surprising findings

• Using DINOv2 as teacher gives much better generative gFID (5.16) than InternViT (7.38) despite InternViT's better reconstruction, showing the encoder's latent structure matters more than the adaptation framework for generation.
• Pure flow-matching loss without distillation (lambda_d:lambda_f = 0:1) collapses MME-P to 817.2, showing the distillation term is essential for understanding.

Limitations

• Generation results are limited: class-conditional ImageNet generation uses only MAR-L and T2I uses a small 0.6B model trained on 1M synthetic pairs.
• Generative quality depends heavily on the teacher encoder's latent properties; InternViT alone gives weaker gFID than DINOv2.
• Several tables mix settings (different resolutions, training data, LLM sizes), so cross-method comparisons are not fully controlled.

Experimental setup

Datasets: ImageNet-1K MS-COCO 2017 LLaVA-v1.5 Pretrain-558K LLaVA-v1.5 Instruct-665K LLaVA-OneVision subset NYU-Depth-v2 ADE20K MIX-5

Benchmarks: ImageNet-1K MS-COCO 2017 MMVet POPE VQAv2 GQA TextVQA MMBench MME GenEval DPG-Bench NYU-Depth-v2

Metrics: PSNR SSIM rFID gFID IS top-1 accuracy AP RMSE mIoU GenEval

Baselines: UniTok TokenFlow VILA-U QLIP SemHiTok BLIP3-o SD-VAE SANA-0.6B Show-o Janus-Pro

Builds on: Rectified Flow VA-VAE LLaVA-v1.5 LLaVA-OneVision MAR DC-Gen

Closest papers by meaning

• TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.93)
• UniTok: a Unified Tokenizer for Visual Generation and Understanding (NeurIPS 2025 · similarity 0.90)
• Image Understanding Makes for A Good Tokenizer for Image Generation (NeurIPS 2024 · similarity 0.89)
• Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.89)
• G$^2$TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models (NeurIPS 2026 · similarity 0.89)
• End-to-End Vision Tokenizer Tuning (NeurIPS 2025 · similarity 0.89)