Research map / Image generation / Text-to-image diffusion
Enhancing Creative Generation on Stable Diffusion-based Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes C3, a training-free method that amplifies low-frequency U-Net features in the down and middle blocks of Stable Diffusion-based models to boost creative generation.
Problem
Stable Diffusion and its distilled variants produce limited creative outputs; adding "creative" to prompts rarely yields genuinely creative images, and prior creativity methods need extra optimization or reference images.
Approach
Amplify low-frequency components of intermediate U-Net features in the shallow down and middle blocks during denoising, with per-block amplification factors chosen automatically by a usability-constrained novelty search (aesthetic plus CLIP score), requiring no training.
Key results
- Across SDXL, Turbo and Lightning variants, C3 raises LPIPS diversity (e.g., Turbo 0.22 to 0.36) and Vendi scores while FID increases (e.g., Turbo 146 to 164), indicating greater novelty.
- User study on Lightning (1-step) shows novelty rising from 2.65 to 4.12 (5-point Likert) with usability dropping from 4.62 to 4.19.
- C3 novelty score in the user study (3.79 for Turbo) surpasses ConceptLab (3.65), while C3 usability (4.14) is far higher than ConceptLab (2.97).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add C3 low-frequency feature amplification | Lightning (1-step) · Five objects, averaged | FID: 123.95±44.94 -> 163.01±58.01 | Helped |
| add C3 low-frequency feature amplification | Turbo · Five objects, averaged | Recall: 0.27±0.17 -> 0.68±0.18 | Helped |
| add C3 low-frequency feature amplification | Lightning (4-step) · Five objects, averaged | Recall: 0.92±0.05 -> 0.83±0.09 | Hurt |
| add C3 low-frequency feature amplification | SDXL · Five objects, averaged | Recall: 0.96±0.01 -> 0.93±0.04 | Hurt |
| add C3 low-frequency feature amplification | Lightning (1-step) · Five objects, user study | Novelty: 2.65 -> 4.12 | Helped |
| add C3 low-frequency feature amplification | Lightning (1-step) · Five objects, user study | Usability: 4.62 -> 4.19 | Hurt |
| remove “creative” from the prompt while applying C3 (ablation) | C3 + Stable Diffusion-based models · Image generation | FID | Hurt |
| amplify all frequency bands instead of only low-frequency features (ablation) | C3 feature amplification · Image generation | Image quality | Hurt |
| amplify low-frequency features only (ablation) | C3 feature amplification · Image generation | Image quality | Helped |
| amplify first and second down-block features (ablation) | Stable Diffusion U-Net · Image generation | Creative visual variation | Helped |
| amplify third down block and middle-block features (ablation) | Stable Diffusion U-Net · Image generation | Creative visual variation | Helped |
| amplify up-block features (ablation) | Stable Diffusion U-Net · Image generation | Image quality | Hurt |
Surprising findings
- Removing the word 'creative' from prompts with C3 yields ordinary images without artifacts and even a slightly lower FID, suggesting C3 amplifies creativity only when the prompt explicitly includes the creative term.
Limitations
- Relies on the generation capacity of the pre-trained model; cannot create creativity the model lacks.
- Effectiveness on transformer-based architectures (e.g., HunYuan-DiT) requires more analysis.
- Precision and recall decline and usability metrics drop relative to original models.
Experimental setup
Benchmarks: Creative prompts on chair, building, garment, car, teddy bear
Metrics: FID Precision Recall LPIPS Vendi CLIP score BLIP VQA score user study Likert GPT-4o creativity type classification
Baselines: ConceptLab Real-to-Ref FreeU SDXL SDXL-Turbo Lightning
Builds on: FreeU ConceptLab ControlNet
Closest papers by meaning
- An analytic theory of creativity in convolutional diffusion models (ICML 2025 · similarity 0.89)
- Redefining <Creative> in Dictionary: Towards an Enhanced Semantic Understanding of Creative Generation (CVPR 2025 · similarity 0.88)
- Seeking the Unfamiliar but Memorable: Conceptual Creativity as Meta-Learning (NeurIPS 2026 · similarity 0.88)
- Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation (ICLR 2024 · similarity 0.88)
- Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion (ICML 2025 · similarity 0.88)
- DreamDistribution: Learning Prompt Distribution for Diverse In-distribution Generation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.