Research map / Image generation / Text-to-image diffusion
PICD: Versatile Perceptual Image Compression with Diffusion Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PICD, a perceptual codec that losslessly encodes OCR text, compresses the image with MLIC plus a glyph-conditioned branch, and renders text back with a fine-tuned Stable Diffusion via a hybrid ControlNet/StableSR adaptor and instance-level guidance.
Problem
Perceptual image codecs produce visually plausible but incorrect text when compressing screen content at low bitrates, while screen content codecs sacrifice perceptual quality.
Approach
Encode text losslessly (OCR text plus coordinates) and compressed image separately, then render text onto the decoded image with a conditional diffusion model, using domain-level finetuning on screenshots, a hybrid ControlNet/StableSR adaptor, and instance-wise OCR and recompression guidance during sampling.
Key results
- On SCI1K, PICD reaches BD-TEXT 0.107 and BD-FID -20.68 versus PerCo's -0.057 and -19.90, the best text accuracy among compared codecs.
- Ablation on screen images: the full model reaches text accuracy 0.4568 and FID 34.77, versus 0.3468 and 45.83 for the basic ControlNet implementation.
- On Kodak, PICD achieves the lowest BD-FID (-74.55) among perceptual codecs, though with lower BD-PSNR than MS-ILLM-style baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PICD instead of MLIC | Screen image compression · SCI1K | BD-TEXT: 0.000 -> 0.107 | Helped |
| Use PICD instead of PerCo | Screen image compression · SCI1K | BD-FID: -19.90 -> -20.68 | Helped |
| Use PICD instead of Text-Sketch | Screen image compression · SIQAD | BD-TEXT: 0.014 -> 0.086 | Helped |
| Use PICD instead of PerCo | Screen image compression · SIQAD | BD-FID: -52.47 -> -52.93 | Helped |
| Use PICD instead of PerCo | Natural image compression · Kodak | BD-FID: -71.29 -> -74.55 | Helped |
| Use PICD instead of PerCo | Natural image compression · CLIC | BD-FID: -48.86 -> -61.35 | Helped |
| Add glyph conditioning to diffusion rendering (ablation) | PICD · Screen images | Text Acc: 0.3468 -> 0.4404 | Helped |
| Replace ControlNet with StableSR (ablation) | PICD adaptor · Screen images | Text Acc: 0.4404 -> 0.3934 | Hurt |
| Use the proposed hybrid adaptor instead of ControlNet or StableSR (ablation) | PICD · Screen images | FID: 45.35 / 49.76 -> 37.90 | Helped |
| Add domain-level fine-tuning (ablation) | PICD diffusion model · Screen images | FID: 49.76 -> 39.81 | Mixed |
| Add instance-level guidance (ablation) | PICD diffusion rendering · Screen images | FID: 37.90 -> 35.54 | Helped |
| Add both domain-level fine-tuning and instance-level guidance (ablation) | PICD · Screen images | Text Acc: 0.4445 -> 0.4568 | Helped |
Surprising findings
- Direct text rendering achieves slightly higher text accuracy than diffusion rendering (0.463 vs 0.445) but much worse FID and CLIP similarity.
Limitations
- Decoding is slow (~30 s) due to instance-level guidance.
- Text accuracy on screen images is still below 0.5 Jaccard at the tested bitrates.
- Relies on Tesseract OCR for text extraction, so OCR errors propagate into the reconstruction.
Experimental setup
Datasets: WebUI OpenImages SCI1K SIQAD Kodak CLIC
Benchmarks: SCI1K SIQAD Kodak CLIC
Metrics: Jaccard text accuracy BD-PSNR BD-FID BD-CLIP BD-LPIPS BD-DISTS
Baselines: MLIC VTM Text-Sketch CDC MS-ILLM PerCo
Builds on: Stable Diffusion ControlNet StableSR MLIC PerCo
Closest papers by meaning
- Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder (ICML 2024 · similarity 0.90)
- One-Step Diffusion-Based Image Compression with Semantic Distillation (NeurIPS 2025 · similarity 0.89)
- Decouple Distortion from Perception: Region Adaptive Diffusion for Extreme-low Bitrate Perception Image Compression (CVPR 2025 · similarity 0.89)
- Towards image compression with perfect realism at ultra-low bitrates (ICLR 2024 · similarity 0.88)
- Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models (ICLR 2025 · similarity 0.88)
- Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.