Research map / Image generation / Text-to-image diffusion

Learning to Generate Multiple Objects from Dense and Occluded Layouts

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Layout-to-image control · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes layout-aware attention biases and an amodal-balanced loss (AIBL) that upweights occluded instances, improving object counting in dense overlapping scenes for a DiT-based text-to-image model, and introduces the OverlapDepth-45K benchmark.

Problem

Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances merge and heavily occluded objects receive weak training gradients.

Approach

Injects depth-ordered region-identity attention biases (cohesion within instance, separation across instances, boosted in overlap zones) into DiT attention logits, and reweights the per-pixel flow-matching loss by an amodal-to-visible area ratio per instance, mean-normalized to keep gradient scale fixed.

Key results

• On OverlapDepth-45K, AIBL reaches 15.32% exact match and RMSE 5.99, versus 3DIS at 12.88% and RMSE 6.34 among layout baselines.
• Adding AIBL to basic layout attention reduces RMSE from 9.26 to 5.99 (reported 35.3% reduction).
• On the T2I-Compbench counting subset (100-image human eval), the method reaches 65% accuracy versus 48% for CountGen.

Surprising findings

• The layout-only baseline marginally outperforms AIBL in the highest density bin (23+ objects), 8.7% vs 8.3%.
• Setting gamma=1.0 causes RMSE to approach the layout-only baseline, showing that stronger occlusion boosting hurts.

Limitations

• Density saturation: latent resolution limits force feature overlap in extremely packed scenes.
• Category bias: thin or articulated objects (e.g., umbrellas) are harder to render.
• OverlapDepth-45K is entirely synthetic, risking domain shift to real images.

Experimental setup

Datasets: OverlapDepth-45K MS-COCO Visual Genome

Benchmarks: OverlapDepth-45K T2I-Compbench CoCoCount

Metrics: Exact Match accuracy relative tolerance accuracy (±10%, ±20%) MAE RMSE counting score

Baselines: FLUX.2-Klein-4b SDXL-Turbo GLIGEN MIGC 3DIS SDXL Repeated Object Reason Out Your Layout DALL-E 3 Random masks + BoundedAttn Counting Guidance CountGen (Make-It-Count)

Builds on: GLIGEN MIGC 3DIS rectified flow matching FLUX.2-Klein

Closest papers by meaning

• OverLay++: Dense-Overlap Layout-to-Image Generation Dataset (NeurIPS 2026 · similarity 0.88)
• OverLayBench: A Benchmark for Layout-to-Image Generation with Dense Overlaps (NeurIPS 2025 · similarity 0.87)
• InstanceDiffusion: Instance-level Control for Image Generation (CVPR 2024 · similarity 0.86)
• Disentangled 3D Scene Generation with Layout Learning (ICML 2024 · similarity 0.86)
• InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attention (NeurIPS 2025 · similarity 0.86)
• ROICtrl: Boosting Instance Control for Visual Generation (CVPR 2025 · similarity 0.85)