Research map / Image generation / Editing and style transfer
One-Shot Structure-Aware Stylized Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OSASIS, a one-shot diffusion-based image stylization method that fine-tunes a pretrained DDIM with CLIP directional losses and uses a structural latent (encoded at a chosen timestep) plus a DiffAE semantic latent to preserve input structure while transferring style.
Problem
One-shot image stylization often fails to preserve the structure of input images, especially for rare, out-of-distribution structures, and GAN-based and existing diffusion-based methods entangle structure with style.
Approach
Disentangle structure from semantics by encoding the input into a structural latent at a chosen timestep with DDIM-style forward encoding conditioned on a DiffAE semantic code, add a structure-preserving 1x1-conv network (SPN) to recover spatial detail, and fine-tune a copy of a pretrained DDPM with cross-domain, in-domain and reconstruction losses using a single style image.
Key results
- On low-density images, OSASIS achieves ArtFID 34.89 on AAHQ, best among compared methods (MTG+HFGI 36.39), and ID similarity 0.6825 vs DiffuseIT 0.6992 on AAHQ.
- On MetFaces, OSASIS reaches ID similarity 0.7323 and ArtFID 43.20, versus 0.7158 for DiffuseIT in ID similarity.
- SPN with lambda=0.1 improves ArtFID from 36.41 to 34.89 and ID similarity from 0.6595 to 0.6825 relative to no SPN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SPN with λSPN=0.1 instead of omitting SPN (ablation) | OSASIS · stylization evaluation | ArtFID: 36.41 -> 34.89 | Helped |
| Add SPN with λSPN=0.1 instead of omitting SPN (ablation) | OSASIS · stylization evaluation | ID similarity: 0.6595 -> 0.6825 | Helped |
| Add SPN with λSPN=0.1 instead of omitting SPN (ablation) | OSASIS · stylization evaluation | Structure distance: 0.0371 -> 0.0361 | Helped |
| Increase SPN weight from λSPN=0.1 to λSPN=0.5 (ablation) | OSASIS · stylization evaluation | ArtFID: 34.89 -> 36.62 | Hurt |
| Increase SPN weight from λSPN=0.1 to λSPN=0.5 (ablation) | OSASIS · stylization evaluation | ID similarity: 0.6825 -> 0.7177 | Helped |
| Increase SPN weight from λSPN=0.1 to λSPN=0.5 (ablation) | OSASIS · stylization evaluation | Structure distance: 0.0361 -> 0.0348 | Helped |
| Increase SPN weight from λSPN=0.1 to λSPN=1.0 (ablation) | OSASIS · stylization evaluation | ArtFID: 34.89 -> 43.15 | Hurt |
| Increase SPN weight from λSPN=0.1 to λSPN=1.0 (ablation) | OSASIS · stylization evaluation | ID similarity: 0.6825 -> 0.7321 | Helped |
| Increase SPN weight from λSPN=0.1 to λSPN=1.0 (ablation) | OSASIS · stylization evaluation | Structure distance: 0.0361 -> 0.0355 | Helped |
Surprising findings
- MTG achieves a better ArtFID than OSASIS on some datasets, and DiffuseIT is stronger on structure distance on AAHQ/MetFaces.
Limitations
- Training takes longer than comparison methods.
- Requires separate training for each style image.
- Structure-preserving network weight lambda must be carefully calibrated; values above 0.1 hurt stylization quality.
Experimental setup
Datasets: FFHQ AAHQ MetFaces AFHQ-dog LSUN-church DeepFashion
Benchmarks: AAHQ MetFaces AFHQ-dog LSUN-church DeepFashion
Metrics: ArtFID ID similarity (ArcFace) structure distance LPIPS
Baselines: MTG+HFGI JoJoGAN+HFGI DiffuseIT InST StyleGAN-NADA
Builds on: DiffAE MTG DiffusionCLIP DDIM JoJoGAN
Closest papers by meaning
- Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and Styles (NeurIPS 2025 · similarity 0.87)
- Conditional Balance: Improving Multi-Conditioning Trade-Offs in Image Generation (CVPR 2025 · similarity 0.86)
- ReGS: Reference-based Controllable Scene Stylization with Gaussian Splatting (NeurIPS 2024 · similarity 0.86)
- Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylization (CVPR 2025 · similarity 0.86)
- StyO: Stylize Your Face in Only One-Shot (AAAI 2025 · similarity 0.85)
- OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Data (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.