Research map / Vision-language and multimodal / Multimodal LLMs

Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Thinking-while-Generating (TwiG), interleaving textual reasoning with autoregressive visual generation of Janus-Pro-7B across three regions, evaluated via zero-shot prompting, SFT on TwiG-50K, and GRPO-based RL on T2I-CompBench.

Problem

Existing chain-of-thought approaches to visual generation apply textual reasoning only before or after synthesis, lacking on-the-fly reasoning that guides and reflects on the image as it is being generated.

Approach

Interleave textual thoughts into the autoregressive visual token stream: each region's thought guides the next region, and a reflection step scores and locally regenerates the just-produced region within a single trajectory, with the three stages (zero-shot, SFT, RL) compared as candidate routes.

Key results

• Zero-shot TwiG-ZS raises Janus-Pro-7B Complex score from 35.59 to 48.16 on T2I-CompBench and Color from 63.59 to 73.11.
• TwiG-SFT reaches Complex 53.41 and Shape 52.42, with std reduced versus TwiG-ZS.
• TwiG-RL achieves Color 82.49, Shape 61.28, Texture 73.19 and Complex 54.45 on T2I-CompBench, with Spatial 34.06 versus 27.02 for SFT.

Surprising findings

• Adding reflection data during SFT degraded performance, as thoughts became longer and over-corrections increased.
• Two rounds of reflection gave no further gain over one round in zero-shot setting.

Limitations

• Fixed three-step schedule; adaptive scheduling not reliably followed by current ULMs.
• Uses original GRPO without recent variants.
• Evaluated only on text-to-image; video, 3D and image-to-image not explored.

Experimental setup

Datasets: T2I-CompBench TwiG-50K

Benchmarks: T2I-CompBench T2I-CompBench++

Metrics: Color Shape Texture Spatial Non-Spatial Complex Numeracy

Baselines: Janus-Pro-7B Think-before-generation Think-after-generation GoT T2I-R1 FLUX.1 SD-XL-base-1.0 PixArt-alpha Show-o Emu3 Attend-and-Excite Show-o + PARM

Builds on: GRPO Janus-Pro Generation with CoT T2I-CompBench

Closest papers by meaning

• ThinkGen: Generalized Thinking for Visual Generation (CVPR 2026 · similarity 0.92)
• GoT-R1: Unleashing Reasoning Capability of Autoregressive Visual Generation with Reinforcement Learning (ICLR 2026 · similarity 0.92)
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.90)
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.90)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.90)
• T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT (NeurIPS 2025 · similarity 0.90)