Research map / Vision-language and multimodal / Human and embodied modeling

Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

NeurIPS 2026 · Vision-language and multimodal › Human and embodied modeling · approach: Vision-language pretraining · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Tuna-2, an encoder-free unified multimodal model that performs image understanding and pixel-space flow-matching generation directly from raw pixel patches, with masked feature learning, and compares it to an encoder-based variant Tuna-R and latent-space Tuna.

Problem

Unified multimodal models rely on pretrained vision encoders and VAE latents, causing representation mismatch between understanding and generation and limiting fine-grained visual perception; the paper asks whether UMMs can be built end-to-end directly from raw pixels.

Approach

Remove both the VAE and the pretrained representation encoder: a single transformer decoder (Qwen2.5-7B-Instruct) consumes raw image patch embeddings for understanding, and a pixel-space flow-matching head with x-prediction and v-loss generates images, with masked patch learning to regularize pixel-space representations.

Key results

• Tuna-2 (7B) reports the best results among native UMMs on several understanding benchmarks, e.g., MMVP 77.3, OCRBench 79.7, V* 59.2, CountBench 81.7, and outperforms Tuna-R on most understanding benchmarks.
• On GenEval, Tuna-2 reaches 0.87 overall versus 0.88 for Tuna-R and 0.90 for Tuna; on DPG-Bench Tuna-2 scores 86.54, slightly above Tuna-R's 86.35.
• In LLM-judge evaluation, Tuna-2 receives 48.4% diversity preference under GPT-5.4 versus 30.9% for Tuna-R, while quality is comparable to Tuna-R (32.1% vs 35.7%).

Surprising findings

• Tuna-R (encoder-based) is ahead of Tuna-2 early in training on understanding benchmarks, but Tuna-2 overtakes it with more data.
• Masking benefits Tuna-2 more than Tuna-R, which the authors hypothesize relates to SigLIP 2's own masked pretraining.

Limitations

• Generation trails the encoder-based Tuna-R on GenEval throughout training and after SFT is only nearly identical; Tuna-2 is slightly behind Tuna on image editing (ImgEdit 4.09 vs 4.31).
• Training uses 550M in-house proprietary image-text pairs, limiting reproducibility.
• LLM-judge evaluation uses only 1.5K prompts and relative preference among three models.

Experimental setup

Datasets: in-house image-text pairs (550M) Nemotron FineVision OmniEdit ImageNet

Benchmarks: GQA RealWorldQA MMVet MMMU MMVP SEED-Bench2+ AI2D ChartQA OCRBench V* CountBench VisuLogic

Metrics: accuracy GenEval overall DPG-Bench overall ImgEdit total score pairwise preference rate (quality, diversity) reconstruction quality

Baselines: Tuna Tuna-R Show-o2 Janus-Pro BAGEL Qwen2.5-VL Mogao OmniGen2 UniWorld FLUX.1 [dev]

Builds on: Tuna JiT rectified flow LLaVA Mono-InternVL

Closest papers by meaning

• UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.88)
• Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs (NeurIPS 2024 · similarity 0.88)
• Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.87)
• Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.87)
• Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation (ICML 2026 · similarity 0.87)
• CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.87)