Research map / Vision-language and multimodal / Multimodal LLMs

Wan-Weaver: Interleaved Multi-modal Generation via Decoupled Training

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Wan-Weaver decomposes interleaved text-image generation into a VLM-based planner (trained on synthetic textual-proxy interleaved data) and a DiT visualizer (trained on reference-guided image data), achieving near-Nano Banana results on OpenING without real interleaved data.

Problem

Interleaved text-image generation is hard because real interleaved training data is scarce and long-range cross-modal coherence is difficult to model; unified models typically output single modalities.

Approach

Decouple interleaved generation into a planner (VLM expert that emits dense textual prompts for images, trained on textual-proxy interleaved data) and a visualizer (DiT trained from scratch on reference-guided generation data with frozen planner features), plus a Dense Prompt Context Window that restricts visualizer attention around the dense prompt position.

Key results

• On OpenING, Wan-Weaver scores 8.67 overall, above GPT-4o+DALL-E3 (8.20) and close to Nano Banana (8.85), and surpasses Nano Banana on Completeness (9.41 vs 9.34).
• On single-modality tasks, Wan-Weaver reaches GenEval 0.89 and DPG 87.21, and ImgEdit 4.31 and GEdit-EN 7.39, the best among listed unified models.
• Planner reaches MMMU 74.9 and MathVista 84.3, close to its Qwen2.5-VL-32B-Think base (75.1 and 84.7).

Surprising findings

• Joint planner-visualizer training causes unstable visualizer loss (around 0.25) compared with decoupled training (down to 0.15).
• Nano Banana's strong coherence and multi-step consistency may stem from repeated or highly similar image outputs.

Limitations

• Visualizer trained from scratch, with large compute (9.6T tokens) implied; ablations run only on 7B variant due to computational limits.
• Evaluation relies on GPT-4o judging and a self-constructed benchmark (WeaverBench) that is not released.
• Visualizer features and planner are fixed in decoupled stages, and the approach is not yet shown to match real interleaved training on all dimensions.

Experimental setup

Datasets: public text-to-image datasets public image editing datasets video key frames with VLM captions SigLIP-clustered homologous image references textual-proxy interleaved data synthesized by LLMs/VLMs

Benchmarks: OpenING WeaverBench MMMU MathVista GenEval DPG ImgEdit GEdit-EN

Metrics: Completeness Quality Richness Correctness Human Alignment IT Coherency Multi-step Consistency Overall Prompt Adherence Narrative Coordination

Baselines: Nano Banana GPT-4o+DALL-E3 Gemini+Flux Emu3 SEED-X VILA-U Show-o Anole BAGEL UniWorld-V1 FLUX.1-dev Step1X-Edit

Builds on: Qwen2.5-VL Wan2.2 mixture-of-transformers flow matching

Closest papers by meaning

• DuoGen: Towards Autonomous Interleaved Multimodal Generation (CVPR 2026 · similarity 0.90)
• WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation (CVPR 2026 · similarity 0.89)
• LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model (NeurIPS 2026 · similarity 0.89)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.89)
• Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation (CVPR 2026 · similarity 0.89)
• Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization (NeurIPS 2025 · similarity 0.88)