Research map / Image generation / Editing and style transfer
Predicting the Original Appearance of Damaged Historical Documents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Historical Document Repair (HDR), a HDR28K dataset of 28,552 synthetically damaged/repaired historical document pairs, and DiffHDR, a diffusion-based repair network with semantic/spatial conditioning and a character perceptual loss.
Problem
Historical documents suffer character missing, paper damage, and ink erosion, but existing document processing methods (binarization, enhancement, etc.) do not repair them; no dataset exists for predicting their original appearance.
Approach
Frame document damage repair as conditional diffusion that takes the damaged patch plus target character content image and character mask as conditions, and adds a VGG-based character perceptual loss restricted to the masked character regions to preserve character content and style.
Key results
- DiffHDR reaches FID 0.7499, LPIPS 0.0384 and Rec-ACC 81.92% on HDR28K, beating the next-best NAFNet (FID 0.8588, LPIPS 0.0435) and Uformer-Big (Rec-ACC 69.95%).
- Removing the character perceptual loss raises FID from 0.7499 to 0.8416 and drops Rec-ACC from 81.92% to 64.49%.
- DiffHDR trained on synthetic HDR28K data repairs real damaged historical document images collected from the Internet, though no real ground-truth pairs were available for quantitative evaluation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DiffHDR instead of NAFNet | HDR28K repair · HDR28K testing set | FID: 0.8588 -> 0.7499 | Helped |
| use DiffHDR instead of NAFNet | HDR28K repair · HDR28K testing set | LPIPS: 0.0435 -> 0.0384 | Helped |
| use DiffHDR instead of Uformer-Big | HDR28K repair · HDR28K testing set | Rec-ACC: 69.9545 -> 81.9180 | Helped |
| add character perceptual loss (CPLoss) (ablation) | DiffHDR · HDR28K testing set | FID: 0.8416 -> 0.7499 | Helped |
| add character perceptual loss (CPLoss) (ablation) | DiffHDR · HDR28K testing set | LPIPS: 0.0450 -> 0.0384 | Helped |
| add character perceptual loss (CPLoss) (ablation) | DiffHDR · HDR28K testing set | Rec-ACC: 64.4897 -> 81.9180 | Helped |
Surprising findings
- PSNR and SSIM are judged unsuitable for HDR because blurring inflates them, so they were not reported.
- Without semantic and spatial cues (content image and mask), DiffHDR fails to repair severely damaged characters.
Limitations
- Requires user-provided character content and location; automatic prediction of damaged content/location is future work.
- No real damaged-repair pairs for quantitative evaluation; real-document results are qualitative only.
- Fails to repair severe damage without semantic and spatial cues.
Experimental setup
Datasets: MTHv2 M5HisDoc HDR28K
Benchmarks: HDR28K testing set
Metrics: FID LPIPS Rec-ACC
Baselines: UNet Pix2Pix CycleGAN Uformer Restormer NAFNet GRL FocalNet UPOCR
Builds on: DDPM InstructPix2Pix classifier-free guidance LaMa
Closest papers by meaning
- Towards Automated Chinese Ancient Character Restoration: A Diffusion-Based Method with a New Dataset (AAAI 2024 · similarity 0.85)
- Descanning: From Scanned to the Original Images with a Color Correction Diffusion Model (AAAI 2024 · similarity 0.85)
- DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks (CVPR 2024 · similarity 0.84)
- DocNLC: A Document Image Enhancement Framework with Normalized and Latent Contrastive Representation for Multiple Degradations (AAAI 2024 · similarity 0.84)
- Reference-Based Post-OCR Processing with LLM for Precise Diacritic Text in Historical Document Recognition (AAAI 2025 · similarity 0.83)
- Improving Image Restoration through Removing Degradations in Textual Representations (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.