Research map / Image generation / Super-resolution
Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TADiSR, a diffusion-based real-world SR model that fine-tunes text-aware cross-attention and adds joint segmentation decoders to restore text fidelity, trained on a new synthetic full-image text SR dataset (FTSR) plus Real-CE.
Problem
Generative real-world image super-resolution often distorts text structures (malformed strokes, wrong characters), and existing text SR methods work on cropped regions and do not generalize to full images.
Approach
Fine-tune LoRA cross-attention for the word "text" so attention maps localize text regions, feed these maps into a jointly trained text segmentation decoder that exchanges features with the VAE image decoder via Cross-Decoder Interaction Blocks, and use a segmentation-modulated focal gradient loss to couple SR and segmentation in a single pass.
Key results
- On FTSR-TE (x4), TADiSR reaches PSNR 25.49, SSIM 0.736, LPIPS 0.152, FID 32.13 and OCR-A 0.662, the best among compared methods.
- On Real-CE-val (aligned, x4), TADiSR achieves OCR-A 0.882 versus 0.743 for HAT, reported as an 18.7% relative improvement over the second-best method.
- In ablation on FTSR-TE, removing the joint segmentation decoders drops OCR-A from 0.662 to 0.594 (11.4% decrease in OCR accuracy).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TADiSR's text-aware SR and joint segmentation framework | Kolors LDM · FTSR-TE (×4) | OCR-A: 0.596 -> 0.662 | Helped |
| add TADiSR's text-aware SR and joint segmentation framework | Kolors LDM · Real-CE-val (Aligned, ×4) | OCR-A: 0.743 -> 0.882 | Helped |
| add Joint Segmentation Decoders (ablation) | TADiSR · FTSR test | OCR-A: 0.594 -> 0.662 | Helped |
| enable Text-Aware Cross-Attention (ablation) | TADiSR · FTSR test | OCR-A: 0.617 -> 0.662 | Helped |
| add modified Focal Loss (ablation) | TADiSR · FTSR test | OCR-A: 0.629 -> 0.662 | Helped |
Limitations
- Fails on characters with extremely dense, fused strokes, where structure deviates from ground truth.
- Relies on pseudo-GT text masks from SAM-TS and OCR filtering, and the synthetic dataset may not fully match real degradations.
- Real-CE training uses only 337 manually filtered pairs.
Experimental setup
Datasets: FTSR Real-CE TextSeg BTS CTR LSDIR
Benchmarks: FTSR-TE Real-CE-val
Metrics: PSNR SSIM LPIPS FID OCR-A
Baselines: Real-ESRGAN HAT SeeSR SupIR OSEDiff MARCONet DiffTSR
Builds on: latent diffusion model Kolors SupIR LoRA SAM
Closest papers by meaning
- SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning (NeurIPS 2025 · similarity 0.90)
- SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (CVPR 2024 · similarity 0.89)
- PRISM: Prior Rectification and Uncertainty-Aware Structure Modeling for Diffusion-Based Text Image Super-Resolution (NeurIPS 2026 · similarity 0.89)
- Text-guided Explorable Image Super-resolution (CVPR 2024 · similarity 0.88)
- The Power of Context: How Multimodality Improves Image Super-Resolution (CVPR 2025 · similarity 0.88)
- Towards Realistic Data Generation for Real-World Super-Resolution (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.