AI Paper Atlas

Research map / Image generation / Text-to-image diffusion

TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering

Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, Furu Wei

ECCV 2024 · Image generation › Text-to-image diffusion · approach: Layout-to-image control · 99 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fine-tune layout planner on 5k caption-OCR pairs instead of few-shot (ablation)Vicuna-7B layout planner M1 · MARIO-Eval keyword extractionaccuracy: 49.65 -> 64.85Improved
increase fine-tuning data beyond 5k to 100k (ablation)layout planner M1 · MARIO-Eval keyword extractionaccuracy: 64.85 -> 62.87No clear effect
use top-left and bottom-right corners instead of a single point (ablation)TextDiffuser-2 layout encoding · MARIO-EvalOCR accuracy: Center 35.19 -> 57.58; LT 28.32 -> 57.58Improved
character-level tokenization of keywords instead of BPE subword (ablation)TextDiffuser-2 layout encoding · MARIO-EvalOCR accuracy: 15.48 -> 57.58Improved
line-level language-model guidance instead of character-level masks (vs TextDiffuser)TextDiffuser-2 · MARIO-EvalOCR accuracy / F-measure: accuracy 56.09 -> 57.58; F-measure 78.24 -> 75.06Mixed
line-level guidance improves layout aesthetics and style diversityTextDiffuser-2 vs TextDiffuser · Human user studylayout aesthetics vote share (%): 28.43 -> 71.57Improved
fine-tune on text images then generate text-free natural imagesTextDiffuser-2 · MS COCO 10k promptsFID: 23.03 -> 24.06No clear effect
increase max sequence length L from 128 to 256 (ablation)TextDiffuser-2 · training coverage and inference timeinference time (s): 6 -> 7 seconds; coverage 94.0% -> 99.2%Hurt

Later papers that compared with this method

5 later accepted papers report TextDiffuser-2 as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy