Research map / Image generation / Text-to-image diffusion
Composing Parts for Expressive Object Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PartComposer, a training-free method that localizes object parts via attention from a part-conditioned diffusion pass and then generates rich-text-specified part attributes in masked regions of a pretrained Stable Diffusion model.
Problem
Text-to-image diffusion models cannot reliably follow fine-grained part-level attribute details in prompts: adding part details either changes the whole image or is ignored.
Approach
Denoise the object region with a part-prompt U-Net so that cross-attention localizes parts, then run parallel region-masked diffusion per part (Rich-Text style attributes plus color gradient guidance) and blend with base-generation background, all without training.
Key results
- On DeepFashion zero-shot part segmentation, PartComposer reaches 24.7 FG-NMI vs 12.0 for Stable Diffusion and 16.0 for Rich-Text.
- On CUB-200, PartComposer reaches 20.5 FG-NMI and 9.2 FG-ARI, versus 8.0 and 0.6 for Stable Diffusion.
- In a user study (about 3.5k opinions), PartComposer was significantly preferred for localized and text-consistent part generation; LPIPS vs base image was 0.168, lowest among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PartComposer instead of StableDiffusion | zero-shot part segmentation · DeepFashion | FG-NMI: 12.0 -> 24.7 | Helped |
| use PartComposer instead of StableDiffusion | zero-shot part segmentation · CUB200 | FG-NMI: 8.0 -> 20.5 | Helped |
| use PartComposer for part-level generation | Stable Diffusion 1.5 image generation · Part-level image generation | LPIPS: 0.467 -> 0.168 | Helped |
| use PartComposer for part-level generation | Stable Diffusion 1.5 image generation · Part-level image generation | CLIP: 0.183 -> 0.201 | Helped |
| use PartComposer for part-level generation | Stable Diffusion 1.5 image generation · Part-level image generation | Aesthetic: 5.68 -> 5.66 | No clear effect |
| add Null-Text Inversion (ablation) | PartComposer part localization · CUB200 | FG-NMI: 23.1 -> 35.4 | Helped |
| add max-based localization (ablation) | PartComposer part localization · CUB200 | FG-ARI: 2.8 -> 11.0 | Helped |
| add dot-product localization (ablation) | PartComposer part localization · CUB200 | FG-NMI: 23.7 -> 35.4 | Helped |
| add independent text embeddings (ablation) | PartComposer part localization · CUB200 | FG-NMI: 31.2 -> 35.4 | Helped |
| set clustering K = 14 instead of K = 9 (ablation) | PartComposer part localization · CUB200 | FG-NMI: 35.4 -> 35.2 | No clear effect |
| set clustering K = 4 instead of K = 9 (ablation) | PartComposer part localization · CUB200 | FG-NMI: 35.4 -> 20.4 | Hurt |
Surprising findings
- Stable Diffusion with part details appended to the prompt often ignores the part instructions or alters the whole image, and recent models (SD3.5, inpainting, editing methods) also failed to follow exact part instructions in the authors' comparisons.
Limitations
- Part generation is bottlenecked by part localization; incorrectly localized parts yield wrong edits.
- Zero-shot part segmentation is not directly comparable to supervised or unsupervised baselines.
- Localization relies on hyperparameters (delta, epsilon, alpha, T_th) and on the base model's attention quality.
Experimental setup
Datasets: DeepFashion CUB-200
Benchmarks: DeepFashion CUB-200
Metrics: FG-NMI FG-ARI NMI ARI LPIPS CLIP score LAION aesthetic score
Baselines: Stable Diffusion Rich-Text InstructPix2Pix SCOPS Unsup-Parts DFF
Builds on: Rich-Text generation MultiDiffusion Plug-and-Play Null-Text Inversion
Closest papers by meaning
- Generating compositional scenes via Text-to-image RGBA Instance Generation (NeurIPS 2024 · similarity 0.89)
- PartGen: Part-level 3D Generation and Reconstruction with Multi-view Diffusion Models (CVPR 2025 · similarity 0.88)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.87)
- LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.87)
- RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.87)
- TokenCompose: Text-to-Image Diffusion with Token-level Supervision (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.