Research map / Image generation / Text-to-image diffusion
Type-R: Automatically Retouching Typos for Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Type-R is a training-free post-processing pipeline that detects typographic errors in text-to-image outputs, erases unintended text, regenerates missing text boxes with GPT-4o, and corrects misspelled words via a text editor.
Problem
Text-to-image models frequently render text incorrectly (missing, extra, or misspelled words), and improving the generator is not guaranteed to eliminate typos.
Approach
Treat typo fixing as a post-processing retouching problem: match OCR-detected words to prompt words via optimal transport on Levenshtein distance, then erase extra text, insert missing text boxes via VLM layout planning, and iteratively re-render misspelled words with a text-editing model, all without fine-tuning the generator.
Key results
- Type-R w/ Flux reaches OCR accuracy 8.55 and GPT graphic score 7.67 on MARIO-Eval, versus TextDiffuser-2's 7.17 OCR and 4.97 graphic.
- Ablation: Flux OCR accuracy rises from 14.4 (base) to 63.0 with augmentation, layout correction and typo correction combined.
- User study (5 participants, 500 ratings per method) ranks Type-R w/ Flux highest on human scores (6.23 overall).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Type-R post-processing | SD3 text-to-image outputs · MARIO-Eval | OCR accuracy: 55.4 -> 48.6 | Hurt |
| Apply Type-R post-processing | Flux text-to-image outputs · MARIO-Eval | OCR accuracy: 56.2 -> 62.0 | Helped |
| Add prompt augmentation (ablation) | Type-R w/ SD3 · MARIO-Eval | OCR accuracy: 8.6 -> 12.2 | Helped |
| Add layout correction (ablation) | Type-R w/ SD3 with augmentation · MARIO-Eval | OCR accuracy: 12.2 -> 27.4 | Helped |
| Add typo correction (ablation) | Type-R w/ SD3 with augmentation · MARIO-Eval | OCR accuracy: 12.2 -> 20.4 | Helped |
| Add layout correction (ablation) | Type-R w/ Flux with augmentation · MARIO-Eval | OCR accuracy: 28.0 -> 53.8 | Helped |
| Add typo correction (ablation) | Type-R w/ Flux with augmentation · MARIO-Eval | OCR accuracy: 28.0 -> 29.4 | Helped |
Surprising findings
- CLIP score favors TextDiffuser's word-only images, which the authors attribute to CLIP's text-spotting bias rather than better images.
- Naive text-box baselines reach high OCR accuracy at much lower GPT graphic scores, showing OCR accuracy alone is a misleading metric.
Limitations
- Performance depends on external OCR, inpainting, and text-editing models; AnyText produces unstable outlines and brightness artifacts.
- Does not guarantee preservation of reading order when words are split by spaces.
- High computational requirement from stacking multiple backend models.
Experimental setup
Datasets: MARIO-Eval MARIO-LAION
Benchmarks: MARIO-Eval
Metrics: GPT graphic design score GPT content alignment score OCR accuracy FID CLIP score human rating
Baselines: TextDiffuser TextDiffuser-2 Stable Diffusion 3 Flux Dall-E 3 SD 3.5
Builds on: AnyText DeepSolo LaMa TextDiffuser TextDiffuser-2
Closest papers by meaning
- Rich Human Feedback for Text-to-Image Generation (CVPR 2024 · similarity 0.87)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.86)
- Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models (AAAI 2025 · similarity 0.85)
- AnyText: Multilingual Visual Text Generation and Editing (ICLR 2024 · similarity 0.85)
- Discriminative Probing and Tuning for Text-to-Image Generation (CVPR 2024 · similarity 0.85)
- TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.