Research map / Image generation / Text-to-image diffusion
Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?
Xingsong Ye, Yongkun Du, Jiaxin Zhang, Chong Sun, Zhineng Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Investigates whether scene text editing can be trained using only real (non-synthetic) paired image data instead of synthetic paired data and complex disentanglement strategies.
Problem
Scene Text Editing (modifying text content in natural images while preserving style) typically needs paired before/after images, which current methods obtain mainly from synthetic data; this synthetic data struggles to capture the diversity and complexity of natural scenes.
Approach
Uses only real paired data for training scene text editing, arguing that real data suffices to avoid the domain gap of synthetic pairs, as suggested by the title; the specific mechanism is not stated in the provided abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train only on real self-paired data (Shuffle&Crop) instead of synthetic triplets | TextRIRO · Train-on-edits STR proxy avg | accuracy (%): 76.89 -> 86.67 | Improved |
| Use Shuffle&Crop instead of Crop only for straight text (ablation) | TextRIRO · IC13 equal-length | accuracy (%): 77.83 -> 80.16 | Improved |
| Use Crop only instead of Shuffle&Crop for curved text (ablation) | TextRIRO · CUTE equal-length | accuracy (%): 58.42 -> 60.93 | Mixed |
| Remove self-pairing perturbation (identity) (ablation) | TextRIRO · ScenePair | ACC: 85.23 -> 4.37 | Hurt |
| Replace ControlNet injection with DiT-based OminiControl (ablation) | TextRIRO · ScenePair | ACC (%): 85.23 -> 52.89 | Hurt |
| Use p_shuffle 0.7 instead of 0.5 (ablation) | TextRIRO · ScenePair | ACC (%): 80.78 -> 82.42 | Improved |
| Use p_shuffle 0.7 instead of 1.0 (ablation) | TextRIRO · ScenePair | ACC (%): 81.72 -> 82.42 | Improved |
| Pre-train text encoder with STR objective instead of glyph pre-training (ablation) | TextRIRO · ScenePair | ACC / style fidelity | Improved |
| Use SD 2.1 VAE instead of SD 1.5 (ablation) | TextRIRO · ScenePair | reconstruction quality: 28.33 -> 30.31 | Improved |
| Mix TextRIRO 1M data with real data for STR | MAERec · Union14M-Benchmark avg | accuracy (%): 86.09 -> 87.98 | Improved |
| Scale TextRIRO data from 1M to 3M | MAERec · Common Benchmarks avg | accuracy (%): 97.13 -> 96.98 | No clear effect |
| Scale TextRIRO data from 1M to 3M | MAERec · Union14M-Benchmark avg | accuracy (%): 87.98 -> 88.00 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- TextCtrl — TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control (NeurIPS 2024) · better on 4 of 4 shared results
- TextSSR — TextSSR: Diffusion-based Data Synthesis for Scene Text Recognition (ICCV 2025) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control (NeurIPS 2024 · similarity 0.90)
- ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing (NeurIPS 2026 · similarity 0.88)
- DreamText: High Fidelity Scene Text Synthesis (CVPR 2025 · similarity 0.88)
- SteerVTE: Seamless Video Text Editing with Style and Glyph Control (NeurIPS 2026 · similarity 0.87)
- Localizing and Editing Knowledge In Text-to-Image Generative Models (ICLR 2024 · similarity 0.87)
- Text-Driven Image Editing via Learnable Regions (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
