Research map / Vision-language and multimodal / Multimodal LLMs

MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Manzano couples a single shared ViT encoder with continuous and discrete (FSQ) adapters, trains an autoregressive unified LLM (300M to 30B) on text, understanding, and generation tokens, and renders pixels with a flow-matching DiT decoder.

Problem

Unified multimodal models trade off understanding against generation, largely because the LLM must process heterogeneous image tokens (continuous semantic features vs. discrete generation tokens) from separate tokenizers.

Approach

A hybrid vision tokenizer uses one shared ViT with two lightweight adapters, a continuous one for understanding and an FSQ-discrete one for generation, so both token types come from the same semantic space; the LLM uses a single autoregressive next-token objective, and a separate diffusion decoder renders pixels from generated image tokens.

Key results

• Manzano-30B reaches 0.85 GenEval overall and 0.54 WISE overall, and Manzano-3B reaches 0.85 GenEval, outperforming other unified MLLMs on GenEval among those listed (e.g. Bagel-14B 0.82).
• Manzano-3B scores 85.7 on OCRBench and 88.2 on ChartQA; Manzano-30B reaches 96.2 on ScienceQA and 57.8 on MMMU (val).
• Hybrid tokenizer ablation with a 1B LLM: 64.9 General / 66.5 Knowledge / 73.3 Text-rich understanding, versus 62.3 Text-rich for pure-discrete and 72.0 for dual-encoder, with GenEval 77 matching pure-discrete.

Surprising findings

• Pure-discrete tokenizers degrade text-rich understanding heavily (62.3 vs 73.3 for hybrid), suggesting quantization information loss.
• Scaling the image decoder improved structural integrity by +9.9 human score but slightly decreased aesthetic quality.

Limitations

• GenEval and DPG saturate at larger scales, so existing benchmarks may not capture emergent capabilities.
• Aesthetic quality drops when scaling the image decoder.
• Pure-discrete and dual-encoder comparisons use an internal MagViT-2 reproduction and reduced 128x128 generation resolution, which may affect fairness.

Experimental setup

Datasets: CC3M CC12M COYO VeCap DreamO DALLE3-1M BLIP-3o ShareGPT-4o JourneyDB MM1.5 SFT recipe

Benchmarks: SEED-Bench RealWorldQA MMBench AI2D ScienceQA MMMU MathVista ChartQA TextVQA DocVQA InfoVQA OCRBench

Metrics: accuracy GenEval score DPG score WISE score human rating of structural integrity, instruction following,

Baselines: Janus-Pro-7B X-Omni-7B Bagel-14B Harmon-1.5B Blip-3o-4B Emu3-8B MetaQuery-XL-7B Qwen-Image InternVL2.5 Qwen2.5-VL-3B GPT-4o

Builds on: LLaVA MagViT-2 DiT-Air MMDiT CLIP

Closest papers by meaning

• Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.92)
• LMFusion: Adapting Pretrained Language Models for Multimodal Generation (NeurIPS 2025 · similarity 0.92)
• The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models (NeurIPS 2025 · similarity 0.91)
• From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities (ICLR 2025 · similarity 0.91)
• Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization (ICLR 2024 · similarity 0.91)
• UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.91)