Research map / Image generation / Super-resolution
Diffusion-based Blind Text Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffTSR, a dual-diffusion framework (image diffusion model on Stable Diffusion plus a discrete text diffusion model) linked by a Mixture of Multi-modality module, for blind Chinese text image super-resolution.
Problem
Blind super-resolution of degraded Chinese text images, where restored images need both accurate character structure (text fidelity) and realistic styles, which existing methods struggle with under complex strokes and severe degradation.
Approach
Jointly run an image diffusion model (IDM, based on Stable Diffusion) and a discrete text diffusion model (TDM) that exchange conditions at every denoising step via a Mixture of Multi-modality (MoM) module, so text recognition improves image restoration and vice versa.
Key results
- On the synthetic CTR-TSR-Test at x4, DiffTSR reaches 21.85 PSNR, 0.231 LPIPS, 8.482 FID, 0.8350 ACC, outperforming all listed baselines.
- On RealCE at x4 (no fine-tuning on RealCE), DiffTSR reaches 17.49 PSNR, 0.336 LPIPS, 70.59 FID, 0.8475 ACC versus MARCONet at 16.04 PSNR and 0.6638 ACC.
- Ablation on CTR-TSR-Test at x4 shows ACC rising from 0.6342 (IDM only) to 0.8350 with TR, TDM and MoM combined.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use initial text recognition (TR) to provide text prior to IDM (ablation) | IDM · CTR-TSR-Test ×4 | ACC: 0.6342 -> 0.6811 | Helped |
| Use initial text recognition (TR) to provide text prior to IDM (ablation) | IDM · RealCE ×4 | ACC: 0.6953 -> 0.7241 | Helped |
| Add TDM to IDM with initial text recognition (ablation) | IDM with TR · CTR-TSR-Test ×4 | ACC: 0.6811 -> 0.7412 | Helped |
| Add TDM to IDM with initial text recognition (ablation) | IDM with TR · RealCE ×4 | ACC: 0.7241 -> 0.7936 | Helped |
| Add MoM to IDM with TR and TDM (ablation) | IDM with TR and TDM · CTR-TSR-Test ×4 | ACC: 0.7412 -> 0.8350 | Helped |
| Add MoM to IDM with TR and TDM (ablation) | IDM with TR and TDM · RealCE ×4 | ACC: 0.7936 -> 0.8475 | Helped |
| Use DiffTSR instead of the strongest listed baseline | Blind text image super-resolution · CTR-TSR-Test ×2 | ACC: 0.8112 -> 0.8594 | Helped |
| Use DiffTSR instead of the strongest listed baseline | Blind text image super-resolution · CTR-TSR-Test ×4 | ACC: 0.6273 -> 0.8350 | Helped |
| Use DiffTSR instead of the strongest listed baseline | Blind text image super-resolution · RealCE ×2 | ACC: 0.7953 -> 0.9085 | Helped |
| Use DiffTSR instead of the strongest listed baseline | Blind text image super-resolution · RealCE ×4 | ACC: 0.7214 -> 0.8475 | Helped |
Surprising findings
- The IDM-only variant produces text-like characters that do not exist in the Chinese alphabet when no text prior is used.
Limitations
- Text prior from an initial recognizer can be wrong under severe degradation, so the joint loop depends on initialization quality.
- Evaluated mainly on Chinese text at 128x512 resolution with text length up to 24 characters.
Experimental setup
Datasets: CTR CTR-TSR-Train CTR-TSR-Test RealCE
Benchmarks: CTR-TSR-Test RealCE
Metrics: PSNR LPIPS FID ACC NED
Baselines: SRCNN ESRGAN NAFNet TSRN TBSRN TATT MARCONet
Builds on: Stable Diffusion Multinomial diffusion (discrete diffusion for text) Real-ESRGAN degradation TransOCR MARCONet
Closest papers by meaning
- Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders (NeurIPS 2025 · similarity 0.87)
- Text Image Inpainting via Global Structure-Guided Diffusion Models (AAAI 2024 · similarity 0.87)
- Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.86)
- One-Step Effective Diffusion Network for Real-World Image Super-Resolution (NeurIPS 2024 · similarity 0.86)
- One Stone with Two Birds: A Null-Text-Null Frequency-Aware Diffusion Models for Text-Guided Image Inpainting (NeurIPS 2025 · similarity 0.86)
- Text-guided Explorable Image Super-resolution (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.