Research map / Image generation / Text-to-image diffusion
Generative Photomontage
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generative Photomontage composites user-selected regions from a ControlNet image stack by doing a multi-label graph cut in diffusion key-feature space and injecting composite self-attention features during denoising.
Problem
Text-to-image and ControlNet generations vary unpredictably with random seed, so a single output rarely contains every part the user wants; users need a way to combine desirable regions across generated images.
Approach
Treat a same-condition ControlNet image stack as intermediate outputs: segment the stack with a multi-label graph cut over diffusion self-attention key features guided by sparse brush strokes, then blend by injecting composite Q/K/V self-attention features (keeping the model's own query for the base image) during denoising.
Key results
- Ours achieves the highest PSNR (22.46) among compared methods, versus 20.57 for CollageDiffusion and 20.13 for IDP.
- IDP attains the best masked LPIPS (0.104) and SSIM (0.888), while Ours has LPIPS 0.123 and SSIM 0.815.
- In a user survey over 12 scenes, the method was rated best for blending quality by a wide margin and comparable to MasaCtrl+ControlNet for realism.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use feature-space self-attention injection instead of pixel-space blending | Generative Photomontage · 20 test examples | PSNR: 20.13 -> 22.46 | Helped |
| use feature-space self-attention injection instead of pixel-space blending | Generative Photomontage · 20 test examples | Masked LPIPS: 0.104 -> 0.123 | Hurt |
| use feature-space self-attention injection instead of pixel-space blending | Generative Photomontage · 20 test examples | Masked SSIM: 0.888 -> 0.815 | Hurt |
| use feature-space self-attention injection instead of pixel-space blending | Generative Photomontage · 20 test examples | Seam Gradient Score: 0.306 -> 0.339 | Mixed |
| replace composite key and value features with concatenated key and value features (ablation) | Generative Photomontage · 20 test examples | Masked LPIPS: 0.123 -> 0.243 | Hurt |
| replace composite key and value features with concatenated key and value features (ablation) | Generative Photomontage · 20 test examples | Masked SSIM: 0.815 -> 0.677 | Hurt |
| replace composite key and value features with concatenated key and value features (ablation) | Generative Photomontage · 20 test examples | PSNR: 22.46 -> 18.37 | Hurt |
| replace composite key and value features with concatenated key and value features (ablation) | Generative Photomontage · 20 test examples | Seam Gradient Score: 0.339 -> 0.354 | No clear effect |
| use model-generated key and value features for the base image instead of composite features (ablation) | Generative Photomontage · 20 test examples | Masked LPIPS: 0.123 -> 0.268 | Hurt |
| use model-generated key and value features for the base image instead of composite features (ablation) | Generative Photomontage · 20 test examples | Masked SSIM: 0.815 -> 0.669 | Hurt |
| use model-generated key and value features for the base image instead of composite features (ablation) | Generative Photomontage · 20 test examples | PSNR: 22.46 -> 18.85 | Hurt |
| use model-generated key and value features for the base image instead of composite features (ablation) | Generative Photomontage · 20 test examples | Seam Gradient Score: 0.339 -> 0.332 | No clear effect |
Surprising findings
- Injecting the base image's initial query features completely causes suboptimal seams; letting the query adapt within the base mask improves blending.
Limitations
- Results depend on at least one correct candidate per region existing in the generated stack.
- Pixel-space PSNR uses graph-cut composites as a proxy since no ground truth exists.
- Evaluated on only 20 test examples and 12 survey scenes.
Experimental setup
Benchmarks: 20 self-created test examples
Metrics: masked LPIPS masked SSIM PSNR seam gradient score user preference survey
Baselines: Interactive Digital Photomontage Blended Latent Diffusion MasaCtrl + ControlNet CollageDiffusion Cross-Domain Compositing GP-GAN Deep Image Blending BLD+MultiDiffusion SAM (adapted)
Builds on: Interactive Digital Photomontage ControlNet MasaCtrl StyleAligned
Closest papers by meaning
- Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis (CVPR 2025 · similarity 0.87)
- Diffusion Self-Distillation for Zero-Shot Customized Image Generation (CVPR 2025 · similarity 0.86)
- CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation (ICLR 2025 · similarity 0.86)
- R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation (ICLR 2024 · similarity 0.86)
- Generating compositional scenes via Text-to-image RGBA Instance Generation (NeurIPS 2024 · similarity 0.86)
- RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.