Research map / Image generation / Editing and style transfer
SSIMBaD: Sigma Scaling with SSIM-Guided Balanced Diffusion for AnimeFace Colorization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSIMBaD, a diffusion colorizer for anime faces that replaces EDM's log-sigma training and rho-schedule with an SSIM-aligned sigma-space transform phi(sigma)=sigma/(sigma+0.3) plus Euler/Heun trajectory refinement, reporting gains over SCFT and AnimeDiffusion on the Danbooru AnimeFace benchmark.
Problem
Automatic reference-guided colorization of anime face sketches needs structural fidelity and style transfer, but noise schedules in existing diffusion colorizers (DDPM cosine, EDM log-sigma) cause non-uniform perceptual (SSIM) degradation across the trajectory.
Approach
Select a monotonic squash transform phi from a candidate set by how linearly SSIM degradation tracks phi(sigma) (R^2), then use phi* = sigma/(sigma+0.3) for training sampling, noise embedding, and inference schedule in EDM, followed by trajectory refinement that backpropagates an MSE loss through the reverse Euler trajectory.
Key results
- SSIMBaD with trajectory refinement reaches PSNR 18.92 / MS-SSIM 0.8512 / FID 34.98 on same-reference, versus 13.32 / 0.7001 / 135.12 for finetuned AnimeDiffusion and 17.17 / 0.7833 / 43.98 for SCFT.
- On cross-reference, SSIMBaD reports PSNR 15.84, MS-SSIM 0.8207, FID 37.10.
- Among 20+ candidate transforms, sigma/(sigma+0.3) gives the highest R^2 (0.9793 in main table; appendix states 0.9949) for SSIM-vs-phi linearity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the conventional noise schedule with SSIM-aligned sigma-space scaling (ablation) | EDM colorization model · Danbooru AnimeFace same-reference | PSNR: 13.30 -> 15.15 | Helped |
| replace the conventional noise schedule with SSIM-aligned sigma-space scaling (ablation) | EDM colorization model · Danbooru AnimeFace same-reference | FID: 52.18 -> 53.33 | Hurt |
| add EDM backbone (ablation) | AnimeDiffusion baseline · Danbooru AnimeFace same-reference | MS-SSIM: 0.6748 -> 0.6426 | Hurt |
| add EDM backbone (ablation) | AnimeDiffusion baseline · Danbooru AnimeFace same-reference | PSNR: 11.39 -> 13.30 | Helped |
| add trajectory refinement (ablation) | SSIMBaD · Danbooru AnimeFace same-reference | PSNR: 15.15 -> 18.92 | Helped |
| add trajectory refinement (ablation) | SSIMBaD · Danbooru AnimeFace same-reference | FID: 53.33 -> 34.98 | Helped |
| add trajectory refinement (ablation) | SSIMBaD without TPS rotation · Danbooru AnimeFace same-reference | MS-SSIM: 0.8446 -> 0.9190 | Helped |
| add TPS rotation to reference transformation (ablation) | SSIMBaD · Danbooru AnimeFace cross-reference | PSNR | Helped |
| use SSIM-aligned sigma-space scaling instead of candidate transformations (ablation) | noise schedule · SSIM degradation under additive noise | R²: 0.9277 -> 0.9793 | Helped |
| use trajectory refinement rather than no refinement (ablation) | SSIMBaD · Danbooru AnimeFace cross-reference | PSNR: 13.04 -> 15.84 | Helped |
| use trajectory refinement rather than no refinement (ablation) | SSIMBaD · Danbooru AnimeFace cross-reference | FID: 55.18 -> 37.10 | Helped |
| use SSIMBaD with trajectory refinement instead of SCFT | anime face colorization · Danbooru AnimeFace cross-reference | MS-SSIM: 0.7627 -> 0.8207 | Helped |
Surprising findings
- Finetuned AnimeDiffusion improves PSNR and MS-SSIM over its pretrained version but its FID rises sharply to 135.12, a trade-off the authors attribute to the finetuning objective.
- Without TPS rotation, the cross-reference gap is large (PSNR 23.10 same vs 14.00 cross), motivating the added rotation augmentation.
Limitations
- Baselines including the proposed method still struggle to restore fine details such as eye color.
- Cross-reference performance remains noticeably below same-reference performance.
- Evaluation is limited to one anime face dataset at 256x256 resolution.
Experimental setup
Datasets: Danbooru AnimeFace dataset (31,696 training pairs, 579 test samples)
Benchmarks: Danbooru AnimeFace test set (same-reference and cross-reference)
Metrics: PSNR MS-SSIM FID
Baselines: SCFT AnimeDiffusion (pretrained) AnimeDiffusion (finetuned) AnimeDiffusion (EDM backbone)
Builds on: EDM AnimeDiffusion DDPM SCFT
Closest papers by meaning
- MangaNinja: Line Art Colorization with Precise Reference Following (CVPR 2025 · similarity 0.87)
- Free-Lunch Color-Texture Disentanglement for Stylized Image Generation (NeurIPS 2025 · similarity 0.86)
- Color Alignment in Diffusion (CVPR 2025 · similarity 0.85)
- Image Referenced Sketch Colorization Based on Animation Creation Workflow (CVPR 2025 · similarity 0.85)
- Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylization (CVPR 2025 · similarity 0.85)
- StableAnimator: High-Quality Identity-Preserving Human Image Animation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.