Research map / Image generation / Text-to-image diffusion

Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Process

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Compositional text-to-image guidance

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SC-CMJP and the training-free single-pass sampler CO-Jump, which couples text and image unmasking/remasking in masked diffusion via cross-modal attention, evaluated on image editing understanding and maze/nonogram joint generation.

Problem

Concurrent joint text-and-image generation with masked diffusion models: existing parallel samplers do not share within-step cross-modal decisions, and MDMs cannot remask, so cross-modal contradictions persist.

Approach

Modality-specific transition rates are functionals of the other modality's confidence, weighted by cross-modal attention and an entropy-based gate; a ReMDM-style remasking jump retracts committed tokens when cross-modal evidence disagrees, all within one forward pass per step on a frozen MDM.

Key results

• On extended ImgEditBench, the method reaches 1.93 ImgEditBench and 0.369 overall mAP, versus 1.78 and 0.354 for MDM and 0.360 for Qwen3-VL-8B on the same images.
• On JNono-Test500 total joint accuracy, the method scores 0.168 versus 0.138 for MMaDA-Parallel and 0.100 for MDM.
• On JMaze-Test500 total joint accuracy, the method scores 0.432 versus 0.424 for MDM.

Surprising findings

• ReMDM collapses on the maze and nonogram tasks (0.180 and 0.002 total joint accuracy), despite remasking being the intended fix.
• Removing self-correction hurts overall understanding mAP the most (-3.2), while the sampler's NFE scaling is monotonic whereas MMaDA-Parallel degrades with more steps.

Limitations

• Evaluated only on the text-image pair; extension to audio, video, or structured outputs is left to future work.
• Joint accuracy and image-edit scoring rely on Gemini judges and pseudo-grounding of generated images.
• Backbone fine-tuned on the authors' own synthetic corpora generated by proprietary models, limiting reproducibility.

Experimental setup

Datasets: ImgEdit OmniEdit ImgEditBench maze-dataset

Benchmarks: ImgEditBench (extended) JMaze-Test500 JNono-Test500

Metrics: GPT-2 Large perplexity token entropy ImgEditBench score mAP@0.5:0.95 joint accuracy text accuracy image accuracy

Baselines: MDM ReMDM MMaDA-Parallel Qwen3-VL-8B

Builds on: ReMDM MDLM MMaDA-Parallel Lumina-DiMOO

Closest papers by meaning

• Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.89)
• DEEM: Diffusion models serve as the eyes of large language models for image perception (ICLR 2025 · similarity 0.88)
• From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Model (NeurIPS 2026 · similarity 0.88)
• I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models (ICML 2025 · similarity 0.88)
• Diffusion Counterfactual Generation with Semantic Abduction (ICML 2025 · similarity 0.87)
• MMaDA: Multimodal Large Diffusion Language Models (NeurIPS 2025 · similarity 0.87)