Research map / Image generation / Editing and style transfer
FashionR2R: Texture-preserving Rendered-to-Real Image Translation with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FashionR2R, a diffusion-based rendered-to-real fashion image translation framework using positive-domain finetuning, a negative domain embedding, and texture-preserving attention control, plus a 10k-image SynFashion dataset.
Problem
Translating rendered (CG) fashion and human images into photorealistic counterparts while preserving fine-grained clothing textures, given no paired training data.
Approach
Injects real-domain knowledge by finetuning Stable Diffusion on real studio photos, injects rendered-domain knowledge as a negative embedding trained on rendered images (used as guidance in place of the text condition), and preserves garment texture by injecting shallow-layer self-attention queries and keys from a DDIM inversion of the rendered input.
Key results
- On SynFashion, the method reports KID 54.720 vs. 59.238 for the best baseline CUT, and LPIPS 0.067 vs. 0.130 for UNSB.
- On Face Synthetics, KID 73.871 vs. 74.445 for VCT, with LPIPS 0.121 (VCT's 0.096 is better).
- Ablation without TAC raises LPIPS to 0.253 on Face Synthetics, showing texture/content preservation depends on TAC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FashionR2R instead of VCT | Rendered-to-real translation · Face Synthetics | KID: 74.445 (2.273) -> 73.871 (1.973) | Helped |
| use FashionR2R instead of VCT | Rendered-to-real translation · Face Synthetics | LPIPS: 0.096 (0.027) -> 0.121 (0.035) | Hurt |
| use FashionR2R instead of UNSB | Rendered-to-real translation · Face Synthetics | SSIM: 0.818 (0.070) -> 0.831 (0.068) | Helped |
| use FashionR2R instead of CUT | Rendered-to-real translation · SynFashion | KID: 59.238 (1.599) -> 54.720 (1.362) | Helped |
| use FashionR2R instead of UNSB | Rendered-to-real translation · SynFashion | LPIPS: 0.130 (0.040) -> 0.067 (0.025) | Helped |
| use FashionR2R instead of UNSB | Rendered-to-real translation · SynFashion | SSIM: 0.891 (0.054) -> 0.881 (0.055) | Hurt |
| add source DKI (negative domain embedding) (ablation) | FashionR2R · Face Synthetics | KID: 77.376 (2.063) -> 73.831 (1.973) | Helped |
| add source DKI (negative domain embedding) (ablation) | FashionR2R · SynFashion | KID: 58.520 (1.902) -> 54.720 (1.362) | Helped |
| add source DKI (negative domain embedding) (ablation) | FashionR2R · Face Synthetics | SSIM: 0.857 (0.059) -> 0.831 (0.068) | Hurt |
| add target DKI (real-domain fine-tuning) (ablation) | FashionR2R · Face Synthetics | KID: 78.927 (2.134) -> 73.831 (1.973) | Helped |
| add target DKI (real-domain fine-tuning) (ablation) | FashionR2R · SynFashion | KID: 60.186 (1.623) -> 54.720 (1.362) | Helped |
| add target DKI (real-domain fine-tuning) (ablation) | FashionR2R · Face Synthetics | SSIM: 0.845 (0.063) -> 0.831 (0.068) | Hurt |
Surprising findings
- Removing TAC gives the best KID on both datasets (69.349 on Face Synthetics, 51.392 on SynFashion), so realism and content consistency trade off against each other.
- Replacing all self-attention maps reduces realism, motivating control restricted to shallow layers.
Limitations
- DDIM inversion makes inference slow (about one minute for 768x1024 images), so it is not real-time.
- Balancing TAC ratio and denoising strength needs per-image empirical tuning.
- Experiments were limited to SD v1.5 and not SDXL; some societal risks (portrait rights, bias) are noted.
Experimental setup
Datasets: iMaterialist (Fashion) 2019 FGVC Face Synthetics SynFashion
Benchmarks: SynFashion Face Synthetics
Metrics: KID LPIPS SSIM user preference
Baselines: CUT SANTA UNSB VCT
Builds on: Stable Diffusion Textual Inversion DDIM inversion Classifier-free guidance Prompt-to-prompt attention control
Closest papers by meaning
- Cascaded Diffusion Models for Virtual Try-On: Improving Control and Resolution (AAAI 2025 · similarity 0.88)
- Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On (CVPR 2024 · similarity 0.88)
- Controllable Human Image Generation with Personalized Multi-Garments (CVPR 2025 · similarity 0.88)
- DreamFit: Garment-Centric Human Generation via a Lightweight Anything-Dressing Encoder (AAAI 2025 · similarity 0.88)
- GarFast: Realistic and Fast Garment Transfer with a Simplified Parser-Free Approach (AAAI 2025 · similarity 0.87)
- AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.