Research map / Image generation / Image restoration
DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DocRes, a generalist Restormer-based model that unifies five document image restoration tasks (dewarping, deshadowing, appearance enhancement, deblurring, binarization) via input-dependent prior-feature visual prompts (DTSPrompt).
Problem
Document image restoration is usually handled by separate task-specific models, which makes systems complex and forgoes multi-task synergies; existing visual prompt methods need extra exemplar pairs, pretraining, or ViT and fail at high-resolution, variable-size inputs.
Approach
Instead of task-only or learned fixed prompts, build a Dynamic Task-Specific Prompt from prior features extracted from the input image (document mask and coordinates for dewarping, background for deshadowing, background discrepancy for appearance, gradient map for deblurring, Sauvola output/threshold/gradient for binarization), concatenate it with the input, and feed a CNN-based Restormer so it wo
Key results
- DocRes achieves new best results on several dewarping, deshadowing, appearance enhancement and deblurring metrics, e.g., TDD PSNR 27.35 vs 24.00 for the prior best GCDRNet-style baseline shown in Table 1.
- On DIBCO'18 binarization DocRes reaches FM 89.82 and pFM 94.33 vs GDB's 91.09 and 94.57, trailing closely.
- Naive unified training of Restormer degrades performance notably (e.g., DIR300 MS-SSIM 0.5943 vs 0.6185 task-specific); DTSPrompt recovers and surpasses task-specific results on most benchmarks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DTSPrompt to the unified Restormer baseline (ablation) | DocRes · DIR300 | MS-SSIM: 0.6030 -> 0.6264 | Helped |
| Add DTSPrompt to the unified Restormer baseline (ablation) | DocRes · DIR300 | AD: 0.302 -> 0.241 | Helped |
| Add DTSPrompt to the unified Restormer baseline (ablation) | DocRes · DIR300 | LD: 7.77 -> 6.83 | Helped |
| Add DTSPrompt to the unified Restormer baseline (ablation) | DocRes · Kligler et al. | SSIM: 0.8977 -> 0.9005 | Helped |
| Add DTSPrompt to the unified Restormer baseline (ablation) | DocRes · OSR | PSNR: 19.47 -> 21.64 | Helped |
| Use DTSPrompt rather than a fixed prompt (ablation) | DocRes · DIBCO’18 | FM: 77.15 -> 89.82 | Helped |
| Use DTSPrompt rather than a fixed prompt (ablation) | DocRes · DIBCO’18 | pFM: 81.18 -> 94.33 | Helped |
| Use DTSPrompt rather than a fixed prompt (ablation) | DocRes · DIBCO’18 | PSNR: 15.28 -> 19.35 | Helped |
| Train one unified model instead of a task-specific model (ablation) | Restormer baseline · TDD | PSNR: 28.95 -> 25.85 | Hurt |
| Train one unified model instead of a task-specific model (ablation) | Restormer baseline · DIBCO’18 | FM: 76.57 -> 74.33 | Hurt |
| Train one unified model instead of a task-specific model (ablation) | Restormer baseline · RealDAE | PSNR: 24.21 -> 21.75 | Hurt |
| Train one unified model instead of a task-specific model (ablation) | Restormer baseline · DocUNet* | SSIM: 0.7621 -> 0.7620 | No clear effect |
Surprising findings
- Unified training with naive task-agnostic inputs causes large degradation, but a fixed (constant) prompt already improves binarization over task-specific training, attributed to multi-task regularization.
- DocRes trained only on scanned binarization and clean deblurring data generalizes to photographed documents better than DE-GAN and DocDiff, attributed to transfer from the other tasks.
Limitations
- Performance still trails specialized SOTA on several tasks, notably binarization (GDB) and dewarping.
- Prompt fusion is a simple channel concatenation; more sophisticated fusion is left unexplored.
- Prior features rely on hand-crafted extractors (Sauvola, median filter, gradients) whose quality bounds the prompt.
Experimental setup
Datasets: Doc3D DIR300 FSDSRD RDD Jung's dataset Kligler's dataset OSR Doc3DShade RealDAE DocUNet Text Deblur Dataset DIBCO'18
Benchmarks: DIR300 Kligler Jung OSR DocUNet* RealDAE TDD DIBCO'18
Metrics: MS-SSIM LD AD PSNR SSIM F-measure pseudo F-measure
Baselines: DocGeo Li et al. 2023 BGSNet DocShadow UDoc-GAN GCDRNet GDB DE-GAN DocDiff Restormer (task-specific)
Builds on: Restormer ProRes Painter DocGeo
Closest papers by meaning
- Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (CVPR 2024 · similarity 0.85)
- Adapting Text-to-Image Generation with Feature Difference Instruction for Generic Image Restoration (CVPR 2025 · similarity 0.85)
- IRPO: Boosting Image Restoration via Post-training GRPO (NeurIPS 2026 · similarity 0.85)
- DocNLC: A Document Image Enhancement Framework with Normalized and Latent Contrastive Representation for Multiple Degradations (AAAI 2024 · similarity 0.84)
- HRVDA: High-Resolution Visual Document Assistant (CVPR 2024 · similarity 0.84)
- ReFIR: Grounding Large Restoration Models with Retrieval Augmentation (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.