Research map / Image generation / Text-to-image diffusion
ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ResAdapter, a 0.5M-parameter plug-and-play adapter (ResCLoRA convolution LoRA plus ResENorm normalization tuning) that lets Stable Diffusion and SDXL and their personalized derivatives generate unrestricted resolutions and aspect ratios while preserving style.
Problem
Text-to-image diffusion models generate poor-quality images outside their trained resolution, and existing resolution-free methods either need costly multi-pass post-processing or fine-tuning on general data that corrupts the style of personalized models.
Approach
Insert LoRA-style adapters only into the convolution layers of downsampler/upsampler blocks (ResCLoRA) and unfreeze only resnet group normalization (ResENorm), trained once on mixed-resolution LAION-5B data with frozen base weights, so the adapter learns resolution priors without absorbing style, and then plugs into any personalized model.
Key results
- ResAdapter (0.5M parameters) achieves FID 26.89 and CLIP 27.26 at 1024x1024 on LAION-COCO versus MultiDiffusion 37.70 and ElasticDiffusion 27.76, with 0.1322 s latency vs 2.50 s for MultiDiffusion.
- At 256x256, ResAdapter on SD1.4 reaches FID 23.01 vs 54.06 for SD1.4 alone, and ResAdapter's ResENorm-only and ResCLoRA-only variants show lower style FID drift than LoRA (e.g., Realism 2.18/2.29 vs 35.67).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ResAdapter instead of ElasticDiffusion | SD1.4 · LAION-COCO, 256x256 | FID: 23.77 -> 23.01 | Helped |
| Apply ResAdapter instead of SD1.4 | SD1.4 · LAION-COCO, 512x512 | FID: 20.50 -> 20.53 | No clear effect |
| Apply ResAdapter instead of SD1.4 | SD1.4 · LAION-COCO, 512x512 | CLIP Score: 27.33 -> 27.32 | No clear effect |
| Apply ResAdapter instead of ElasticDiffusion | SD1.4 · LAION-COCO, 1024x1024 | latency: 1.16 -> 0.1322 | Helped |
| Add ResENorm to ResCLoRA (ablation) | ResAdapter · Personalized models, resolution generation | image quality | Helped |
| Add ResCLoRA to ResENorm (ablation) | ResAdapter · Personalized models, resolution generation | image quality | Helped |
| Increase alpha_r from 0 to 1 (ablation) | ResAdapter · Lower- and higher-resolution image generation | image quality | Helped |
Surprising findings
- Training all normalization layers on general data causes style conflicts, so only resnet group normalization is unfrozen.
- Style drift from LoRA trained on LAION-5B is large enough that LoRA is reported to change personalized model domains (FID 35-51 vs about 2-4 for ResAdapter variants).
Limitations
- Normalization-layer training on DiT was exploratory and directly training normalization layers can cause style conflicts, so the method is UNet-specific for now.
- Human evaluation uses only 5 annotators and style evaluation uses FID on generated images, which may not fully capture quality.
- Gains in FID at 512x512 are negligible relative to the base SD1.4 (20.53 vs 20.50).
Experimental setup
Datasets: LAION-5B LAION-COCO CivitAI prompts
Benchmarks: LAION-COCO
Metrics: FID CLIP Score latency human preference counts style FID
Baselines: MultiDiffusion ElasticDiffusion SD1.5 SDXL LoRA Diffit
Builds on: LoRA MultiDiffusion ElasticDiffusion Stable Diffusion SDXL
Closest papers by meaning
- ResMaster: Mastering High-Resolution Image Generation via Structural and Fine-Grained Guidance (AAAI 2025 · similarity 0.89)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.88)
- RepLDM: Reprogramming Pretrained Latent Diffusion Models for High-Quality, High-Efficiency, High-Resolution Image Generation (NeurIPS 2025 · similarity 0.88)
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.87)
- Native-Resolution Image Synthesis (NeurIPS 2025 · similarity 0.87)
- Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.