Research map / Image generation / Editing and style transfer
Self-Prompting Diffusion Transformer for Open-Vocabulary Scene Text Edit via In-Context Learning
Hongxi Li, Tong Wang, WU CHENGJING, Tianbao Liu, Jiangtao Yao, Xiaochao Qu, Xinxiao Wu, Luoqi Liu, Ting Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add style prompts and cooldown training on paired data (ablation) | self-prompting MMDiT · AnyText-benchmark English | FID: 11.56 -> 7.62 | Improved |
| add style prompts and cooldown training on paired data (ablation) | self-prompting MMDiT · AnyText-benchmark Chinese | FID: 12.01 -> 7.95 | Improved |
| add style prompts and cooldown training on paired data (ablation) | self-prompting MMDiT · AnyText-benchmark English | Sentence accuracy: 0.8738 -> 0.8857 | Improved |
| add style prompts and cooldown training on paired data (ablation) | self-prompting MMDiT · AnyText-benchmark English | LPIPS: 0.0608 -> 0.0365 | Improved |
| add glyph prompt (rendered target text) (ablation) | MMDiT pretraining · AnyText-benchmark English | Seq. ACC: 0.4127 -> 0.8738 | Improved |
| add style prompt during self-supervised pretraining on top of glyph prompt (ablation) | MMDiT pretraining · AnyText-benchmark English | Seq. ACC: 0.8738 -> 0.5412 | Hurt |
| cooldown with paired data when using glyph and style prompts (ablation) | MMDiT · AnyText-benchmark English | Seq. ACC: 0.5440 -> 0.7982 | Improved |
| cooldown with unpaired data when using glyph prompt only (ablation) | MMDiT · AnyText-benchmark English | Seq. ACC: 0.8738 -> 0.8788 | No clear effect |
| add style prompt without glyph prompt (ablation) | MMDiT pretraining · AnyText-benchmark English | Seq. ACC: 0.4127 -> 0.5236 | Mixed |
| add more languages sequentially to training (ablation) | multilingual MMDiT · MST-Edit | Seq. ACC: Arabic 0.6812 -> 0.7596 | Improved |
| use mask-guided formulation instead of instruction-based general editors | self-prompting MMDiT vs FireRed-Image-Edit-1.1 · AnyText-benchmark English | Seq. Acc: 0.7375 -> 0.8857 | Improved |
| text length above 16 characters | self-prompting MMDiT · text length 2 to 20 characters | visual quality | Hurt |
Closest papers by meaning
- TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control (NeurIPS 2024 · similarity 0.90)
- GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing (CVPR 2025 · similarity 0.90)
- Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.88)
- Textualize Visual Prompt for Image Editing via Diffusion Bridge (AAAI 2025 · similarity 0.88)
- SteerVTE: Seamless Video Text Editing with Style and Glyph Control (NeurIPS 2026 · similarity 0.88)
- RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
