Research map / Image generation / Text-to-image diffusion
TextDiffuser: Diffusion Models as Text Painters
Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, Furu Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add explicit layout and character segmentation guidance to the diffusion model | TextDiffuser · MARIO-Eval | OCR F-measure: 0.5865 -> 0.7824 | Improved |
| Add width embedding (ablation) | Layout Transformer with 1 layer · MARIO-10M test | IoU: 0.268 -> 0.289 | Improved |
| Add width embedding (ablation) | Layout Transformer with 2 layers · MARIO-10M test | IoU: 0.269 -> 0.298 | Improved |
| Add width embedding (ablation) | Layout Transformer with 4 layers · MARIO-10M test | IoU: 0.294 -> 0.297 | Improved |
| Use 2 Transformer layers rather than 1 or 4, with width embedding (ablation) | Layout Transformer · MARIO-10M test | IoU: 0.289, 0.298, 0.297 | Improved |
| Add character-aware loss with weight 0.01 rather than weight 0 (ablation) | TextDiffuser · DrawBenchText | Accuracy: 0.396 -> 0.494 | Improved |
| Set character-aware loss weight to 0.001 rather than 0 (ablation) | TextDiffuser · DrawBenchText | Accuracy: 0.396 -> 0.486 | Improved |
| Set character-aware loss weight to 0.1 rather than 0 (ablation) | TextDiffuser · DrawBenchText | Accuracy: 0.396 -> 0.420 | No clear effect |
| Set character-aware loss weight to 1 rather than 0 (ablation) | TextDiffuser · DrawBenchText | Accuracy: 0.396 -> 0.400 | No clear effect |
| Train whole-image and part-image generation branches together rather than train only one branch (ablation) | TextDiffuser · DrawBenchText and text inpainting | Accuracy / Det-F / Spot-F: 0.494 / 0.380 / 0.218 -> 0.552 / 0.881 / 0.715 | Mixed |
| Use a 0.25 whole-image training ratio rather than 0 (ablation) | TextDiffuser · DrawBenchText whole-image generation | Accuracy: 0.344 -> 0.562 | Improved |
| Use a 0.75 whole-image training ratio rather than 0.5 (ablation) | TextDiffuser · Text inpainting | Det-F: 0.881 -> 0.921 | Improved |
Later papers that compared with this method
11 later accepted papers report TextDiffuser as a baseline in their results tables; 9 of them report a better number on most of the results they share with it.
- POCA: Pareto-Optimal Curriculum Alignment for Visual Text Generation (CVPR 2026 · better on 1 of 1 shared result)
- Type-R: Automatically Retouching Typos for Text-to-Image Generation (CVPR 2025 · better on 3 of 5 shared results)
- HDLayout: Hierarchical and Directional Layout Planning for Arbitrary Shaped Visual Text Generation (AAAI 2025 · better on 2 of 2 shared results)
- GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models (AAAI 2025 · better on 1 of 1 shared result)
- Layout-Agnostic Scene Text Image Synthesis with Diffusion Models (CVPR 2024 · better on 0 of 9 shared results)
- Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model (AAAI 2024 · better on 4 of 5 shared results)
- TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering (ECCV 2024 · better on 4 of 5 shared results)
- TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control (NeurIPS 2024 · better on 3 of 3 shared results)
3 more not listed.
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Layout-Agnostic Scene Text Image Synthesis with Diffusion Models (CVPR 2024 · similarity 0.90)
- DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models (CVPR 2025 · similarity 0.90)
- Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model (AAAI 2024 · similarity 0.89)
- Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models (AAAI 2025 · similarity 0.89)
- Text Image Inpainting via Global Structure-Guided Diffusion Models (AAAI 2024 · similarity 0.89)
- R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
