Research map / Image generation / Text-to-image diffusion
TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering
Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, Furu Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune layout planner on 5k caption-OCR pairs instead of few-shot (ablation) | Vicuna-7B layout planner M1 · MARIO-Eval keyword extraction | accuracy: 49.65 -> 64.85 | Improved |
| increase fine-tuning data beyond 5k to 100k (ablation) | layout planner M1 · MARIO-Eval keyword extraction | accuracy: 64.85 -> 62.87 | No clear effect |
| use top-left and bottom-right corners instead of a single point (ablation) | TextDiffuser-2 layout encoding · MARIO-Eval | OCR accuracy: Center 35.19 -> 57.58; LT 28.32 -> 57.58 | Improved |
| character-level tokenization of keywords instead of BPE subword (ablation) | TextDiffuser-2 layout encoding · MARIO-Eval | OCR accuracy: 15.48 -> 57.58 | Improved |
| line-level language-model guidance instead of character-level masks (vs TextDiffuser) | TextDiffuser-2 · MARIO-Eval | OCR accuracy / F-measure: accuracy 56.09 -> 57.58; F-measure 78.24 -> 75.06 | Mixed |
| line-level guidance improves layout aesthetics and style diversity | TextDiffuser-2 vs TextDiffuser · Human user study | layout aesthetics vote share (%): 28.43 -> 71.57 | Improved |
| fine-tune on text images then generate text-free natural images | TextDiffuser-2 · MS COCO 10k prompts | FID: 23.03 -> 24.06 | No clear effect |
| increase max sequence length L from 128 to 256 (ablation) | TextDiffuser-2 · training coverage and inference time | inference time (s): 6 -> 7 seconds; coverage 94.0% -> 99.2% | Hurt |
Later papers that compared with this method
5 later accepted papers report TextDiffuser-2 as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.
- Type-R: Automatically Retouching Typos for Text-to-Image Generation (CVPR 2025 · better on 3 of 5 shared results)
- DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models (CVPR 2025 · better on 4 of 4 shared results)
- FonTS: Text Rendering With Typography and Style Controls (ICCV 2025 · better on 1 of 1 shared result)
- Glyph-ByT5: A Customized Text Encoder for Accurate Visual Text Rendering (ECCV 2024 · better on 2 of 2 shared results)
- Visual Text Generation in the Wild (ECCV 2024 · better on 1 of 1 shared result)
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- TextDiffuser — TextDiffuser: Diffusion Models as Text Painters (NeurIPS 2023) · better on 4 of 5 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models (CVPR 2025 · similarity 0.90)
- Layout-Agnostic Scene Text Image Synthesis with Diffusion Models (CVPR 2024 · similarity 0.89)
- Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model (AAAI 2024 · similarity 0.89)
- DreamText: High Fidelity Scene Text Synthesis (CVPR 2025 · similarity 0.89)
- Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement (CVPR 2026 · similarity 0.88)
- HDLayout: Hierarchical and Directional Layout Planning for Arbitrary Shaped Visual Text Generation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
