Research map / Image generation / Autoregressive and latent generation
Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DisCo, which disentangles layouts and semantics from a textual scene graph via a VAE (SL-VAE), injects them into Stable Diffusion 1.5 with a masked attention (CMA), and uses a multi-layered sampler for isolated graph edits.
Problem
Scene-graph-to-image generation struggles to faithfully render complex scenes: layout-based methods give one-to-one layouts and miss non-spatial interactions, while semantics-based methods drop independent nodes.
Approach
A Semantics-Layout VAE built on triplet-GCN jointly samples diverse layouts and interactive semantics from one scene graph (one-to-many), and a Compositional Masked Attention inserted into the diffusion model restricts each object's tokens to its own embedding, with a Multi-Layered Sampler for consistent node addition and attribute control.
Key results
- On COCO-Stuff, DisCo reaches IS 23.1 and FID 30.8, versus R3CD at 19.5 and 32.9.
- On Visual Genome, DisCo reaches IS 22.3 and FID 21.9, versus R3CD at 18.9 and 23.4.
- On T2I-CompBench, DisCo scores 0.2376 UniDet and 0.6959 B-VQA, above GORS at 0.1815 and 0.6603.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine layout and interactive-semantics decoders rather than use layout alone (ablation) | SL-VAE · Visual Genome | G2I-ACC: 70.3 -> 73.9 | Helped |
| combine layout and interactive-semantics decoders rather than use layout alone (ablation) | SL-VAE · Visual Genome | I2G-ACC: 70.5 -> 74.3 | Helped |
| add the semantics decoder to independently processed node embeddings (ablation) | SL-VAE · Visual Genome | G2I-ACC: 72.9 -> 73.9 | Helped |
| add the semantics decoder to independently processed node embeddings (ablation) | SL-VAE · Visual Genome | I2G-ACC: 72.8 -> 74.3 | Helped |
| add the attention mask to CMA (ablation) | DisCo · Visual Genome | IS: 17.9 -> 22.3 | Helped |
| add the attention mask to CMA (ablation) | DisCo · Visual Genome | FID: 28.4 -> 21.9 | Helped |
| use separate MLPs for object and attribute embeddings instead of a union MLP (ablation) | CMA · Visual Genome | IS: 19.8 -> 22.3 | Helped |
| use separate MLPs for object and attribute embeddings instead of a union MLP (ablation) | CMA · Visual Genome | FID: 22.0 -> 21.9 | Helped |
| add MLS instead of sampling without MLS (ablation) | DisCo · Visual Genome | IS: 20.5 -> 22.3 | Helped |
| add MLS instead of sampling without MLS (ablation) | DisCo · Visual Genome | FID: 23.0 -> 21.9 | Helped |
| replace randomly scrambled layouts in LSD with SL-VAE layouts and semantics in MLS (ablation) | DisCo · Visual Genome | IS: 21.1 -> 22.3 | Helped |
| replace randomly scrambled layouts in LSD with SL-VAE layouts and semantics in MLS (ablation) | DisCo · Visual Genome | FID: 22.7 -> 21.9 | Helped |
Experimental setup
Datasets: Visual Genome COCO-Stuff
Benchmarks: COCO-Stuff Visual Genome T2I-CompBench
Metrics: Inception Score FID UniDet CLIP score B-VQA 3-in-1 G2I-ACC I2G-ACC ACC_attr
Baselines: SG2Im PasteGAN SOAP WSGC KCGM LDM SGDiff SceneGenie R3CD SD-v1.4 SD-v2 Composable
Builds on: Stable Diffusion triplet-GCN SceneDiffusion SGDiff CVAE
Closest papers by meaning
- R3CD: Scene Graph to Image Generation with Relation-Aware Compositional Contrastive Control Diffusion (AAAI 2024 · similarity 0.91)
- Scene Graph-Grounded Image Generation (AAAI 2025 · similarity 0.90)
- GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs (CVPR 2024 · similarity 0.88)
- Disentangled 3D Scene Generation with Layout Learning (ICML 2024 · similarity 0.88)
- Generating compositional scenes via Text-to-image RGBA Instance Generation (NeurIPS 2024 · similarity 0.88)
- FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Prompts (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.