Research map / Image generation / Image restoration
Human Body Restoration with One-Step Diffusion Model and A New Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds an automated cropping/filtering pipeline (HQ-ACF) to create the PERSONA human-body restoration dataset and proposes OSDHuman, a one-step SD-based restorer whose prompts come from a learned high-fidelity image embedder (HFIE) plus VSD regularization.
Problem
Human body restoration lacks benchmark datasets with high-quality, diverse human images, and existing diffusion restorers (VQVAE- or VAE-based, multi-step or one-step) do not handle human-specific details such as faces and fingers well.
Approach
Mine high-quality person crops from existing object-detection and restoration datasets using detection, Laplacian blur filtering and no-reference IQA ranking; then restore with a one-step SD model whose text prompt is produced by an MHA layer over RAM Swin encoder features with learnable queries (HFIE) instead of RAM tags, trained with MSE, LPIPS and VSD losses.
Key results
- OSDHuman achieves the best PERSONA-Val LPIPS (0.2627), DISTS (0.1414), PSNR (22.41) and FID (16.60) among compared methods, with CLIPIQA 0.7155 and MUSIQ 73.77 on PERSONA-Test.
- PERSONA (109,053 images) scores better than seven human-related datasets on no-reference IQA (e.g., BRISQUE 10.38 vs 15.21 for COCO).
- Using HFIE on LQ inputs for prompts beats RAM-tag DAPE and no-prompt baselines on PERSONA-Test NIQE (4.13 vs 4.95 for DAPE).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train OSDHuman on PERSONA instead of LSDIR (ablation) | OSDHuman · PERSONA-Val | DISTS: 0.1521 -> 0.1414 | Helped |
| Train OSDHuman on PERSONA instead of LSDIR (ablation) | OSDHuman · PERSONA-Val | PSNR: 22.51 -> 22.41 | Hurt |
| Train OSDHuman on PERSONA instead of LSDIR (ablation) | OSDHuman · PERSONA-Test | CLIPIQA: 0.6781 -> 0.7155 | Helped |
| Use HFIE prompts from LQ images instead of no prompt extractor (ablation) | OSDHuman · PERSONA-Test | CLIPIQA: 0.7016 -> 0.7155 | Helped |
| Replace HFIE with DAPE prompts extracted from HQ images (ablation) | OSDHuman · PERSONA-Test | CLIPIQA: 0.6625 -> 0.7155 | Hurt |
| Extract HFIE prompts from HQ instead of LQ images (ablation) | OSDHuman · PERSONA-Test | CLIPIQA: 0.7111 -> 0.7155 | Hurt |
| Use OSDHuman instead of the strongest reported baseline | PERSONA-Val · PERSONA-Val | DISTS: 0.1379 -> 0.1414 | Hurt |
| Use OSDHuman instead of the strongest reported baseline | PERSONA-Val · PERSONA-Val | LPIPS: 0.2756 -> 0.2627 | Helped |
| Use OSDHuman instead of the strongest reported baseline | PERSONA-Test · PERSONA-Test | MUSIQ: 73.5634 -> 73.7694 | Helped |
Surprising findings
- Extracting prompts from HQ images with HFIE hurt NIQE and MUSIQ relative to LQ-based prompts, indicating the LQ-based prompt is preferable at inference.
- Retraining on PERSONA gave only a small PSNR gain over LSDIR training (22.41 vs 22.51 favored LSDIR), while other metrics improved.
Limitations
- Reliance on no-reference IQA metrics for filtering and evaluation may bias toward metric-favored outputs; multi-step methods get higher CLIPIQA in some cases.
- Test set derived from VOC with synthetic-threshold sampling, so 'real-world' LQ labeling is pipeline-dependent.
Experimental setup
Datasets: COCO OID Objects365 CrowdHuman LSDIR VOC DeepFashion iDesigner DIV2K FFHQ
Benchmarks: PERSONA-Val PERSONA-Test
Metrics: DISTS LPIPS PSNR SSIM FID CLIPIQA MANIQA MUSIQ NIQE BRISQUE
Baselines: DiffBIR SeeSR PASD ResShift SinSR OSEDiff
Builds on: OSEDiff SinSR RAM Stable Diffusion LoRA
Closest papers by meaning
- OSDFace: One-Step Diffusion Model for Face Restoration (CVPR 2025 · similarity 0.85)
- PFStorer: Personalized Face Restoration and Super-Resolution (CVPR 2024 · similarity 0.85)
- InstructRestore: Region-Customized Image Restoration with Human Instructions (NeurIPS 2025 · similarity 0.85)
- HAODiff: Human-Aware One-Step Diffusion via Dual-Prompt Guidance (NeurIPS 2025 · similarity 0.84)
- DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation (NeurIPS 2024 · similarity 0.84)
- Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-body (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.