Research map / Detection and segmentation / Open-set and few-shot detection
SHDocs: A dataset, benchmark, and method to efficiently generate high-quality, real-world specular highlight data with near-perfect alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a polarization-based capture method and the SHDocs dataset of real specular highlights on FUNSD document images, plus a benchmark evaluating highlight removal models by image quality and OCR performance.
Problem
Specular highlights occlude text and objects in vision tasks like OCR, but real-world paired (aligned counterfactual) highlight data is scarce and existing synthetic data generalizes poorly.
Approach
Use a polarization sensor with a polarized LED enclosure: highlights are removed by taking per-pixel minimum over four polarization angles to obtain near-aligned deglared counterfactuals, enabling cheap real-world paired data on documents.
Key results
- On SHDocs, only M2-Net beat the no-enhancement baseline in PSNR (32.72 vs 32.18); MIMO-UNetPlus gave the best OCR results, e.g. Textract WER 0.582 vs 0.593 with no enhancement.
- MIMO-UNetPlus retrained on SHDocs reached 42.45 PSNR on SHDocs and 28.0 PSNR on PSD, the highest PSNR on PSD among compared models.
- SHDocs contains 3184 scenes and 19104 images, captured from 199 FUNSD forms with 15 transparency-filtered and 1 unfiltered capture each.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply specular highlight removal instead of no enhancement | Specular highlight removal models on SHDocs · SHDocs evaluation | PSNR: 32.18 -> 32.72 (M2-Net); 32.18 -> 30.48 (Fu et al.); 32.18 -> 30.66 (TSHRNet); 32.18 -> 30.96 (Hu et al.); 32.18 -> 31.6 | Mixed |
| Apply specular highlight removal instead of no enhancement | Specular highlight removal models on SHDocs · SHDocs evaluation | SSIM: 0.9589 -> 0.8599 (Fu et al.); 0.9589 -> 0.9426 (M2-Net); 0.9589 -> 0.9565 (TSHRNet); 0.9589 -> 0.9335 (Hu et al.); 0.958 | Hurt |
| Apply specular highlight removal instead of no enhancement | Specular highlight removal models followed by Textract · SHDocs evaluation | WER: 0.593 -> 0.582 (MIMO-UNetPlus); 0.593 -> 0.591 (TSHRNet); 0.593 -> 0.960 (Fu et al.); 0.593 -> 0.698 (M2-Net); 0.593 -> | Mixed |
| Apply specular highlight removal instead of no enhancement | Specular highlight removal models followed by EasyOCR · SHDocs evaluation | WER: 0.950 -> 0.958 (TSHRNet); 0.950 -> 0.995 (M2-Net); 0.950 -> 0.996 (Hu et al.); 0.950 -> 1.00 (Fu et al.); 0.950 -> 0.942 | Hurt |
| Apply specular highlight removal instead of no enhancement | Specular highlight removal models followed by Tesseract · SHDocs evaluation | WER: 0.846 -> 0.824 (MIMO-UNetPlus); 0.846 -> 0.847 (TSHRNet); 0.846 -> 1.00 (Fu et al.); 0.846 -> 0.933 (M2-Net); 0.846 -> 0 | Mixed |
| Train MIMO-UNetPlus on SHDocs rather than GoPro | MIMO-UNetPlus · PSD | PSNR: 18.1 -> 28.0 | Helped |
| Train MIMO-UNetPlus on SHDocs rather than GoPro | MIMO-UNetPlus · SHIQ | PSNR: 23.1 -> 22.4 | Mixed |
| Train MIMO-UNetPlus on SHDocs rather than GoPro | MIMO-UNetPlus · SHIQ | PSNR: 22.52 -> 23.97 | Helped |
| Train MIMO-UNetPlus on SHDocs rather than GoPro | MIMO-UNetPlus · RD | PSNR: 15.27 -> 16.12 | Helped |
| Train MIMO-UNetPlus on SHDocs rather than GoPro | MIMO-UNetPlus · SHDocs | PSNR: 31.61 -> 42.45 | Helped |
| Train MIMO-UNetPlus on SHIQ rather than SHDocs | MIMO-UNetPlus · SHDocs | PSNR: 33.76 -> 42.45 | Hurt |
Surprising findings
- Image quality metrics (PSNR, SSIM, UIQI) did not track downstream OCR performance; the untrained deblurring MIMO-UNetPlus gave the best OCR results despite not being trained on highlights.
- Most specular highlight removal models scored below the no-enhancement baseline on the image quality metrics.
Limitations
- Polarization-based alignment fails for unpolarized light or polarized light not orthogonal to the sensor, so some highlights remain in the deglared ground truth.
- Dataset covers only FUNSD documents: US-origin, English-only, limited contexts.
- Hardware-specific pipeline (FLIR Blackfly S, Sony IMX250MZR) limits adoption by researchers without that sensor.
Experimental setup
Datasets: FUNSD SHDocs SHIQ PSD RD GoPro
Benchmarks: SHDocs evaluation set SHIQ PSD RD
Metrics: PSNR SSIM UIQI WER CER Levenshtein edit distance
Baselines: Fu et al. SHIQ model M2-Net TSHRNet Hu et al. highlight-aware network MIMO-UNetPlus
Builds on: FUNSD MIMO-UNet
Closest papers by meaning
- Hierarchical Adaptive Filtering Network for Text Image Specular Highlight Removal (CVPR 2025 · similarity 0.86)
- MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis (CVPR 2025 · similarity 0.83)
- PHR-DIFF: Portrait Highlights Removal via Patch-aware Diffusion Model (AAAI 2025 · similarity 0.82)
- Learning to Remove Wrinkled Transparent Film with Polarized Prior (CVPR 2024 · similarity 0.81)
- UPOCR: Towards Unified Pixel-Level OCR Interface (ICML 2024 · similarity 0.81)
- OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.