Research map / Image generation / Autoregressive and latent generation

OverLay++: Dense-Overlap Layout-to-Image Generation Dataset

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Layout generation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces OverLay++, a ~500K-image dense, overlap-aware layout-to-image dataset with short and long global and per-object captions, built with Qwen3-VL-32B detection and per-object validation, and shows gains when training EliGen-SD3 and SiamLayout-SD3 on it.

Problem

Layout-to-Image models struggle with complex scenes containing many overlapping, interacting objects, and existing L2I datasets have sparse layouts (about 1.8-3.9 objects per image) and short, attribute-poor object captions.

Approach

Prompt an MLLM to produce object proposals with deliberately overlapping bounding boxes on LAION-Aesthetics images, validate each object by cropping and asking a binary presence question, then caption at scene and object level in both short and long forms.

Key results

• EliGen-SD3 trained on OverLay++ improves OverLayBench-Complex mIoU from 50.53 to 57.90 and O-mIoU from 22.97 to 30.99.
• On LayoutSAM-Eval, EliGen-SD3 (Ours) improves Color from 82.96% to 90.03% and Shape from 86.35% to 92.81%.
• On DenseLayout, EliGen-SD3 (Ours) raises mIoU from 45.79 to 46.92 and Color from 37.65% to 40.61%.

Surprising findings

• Long local captions (about 89.7 words) degraded EliGen-SD3 performance, which the authors attribute possibly to CLIP 77-token truncation.
• Longer-caption 2K-step ablation worsened CLIP_G and FID, with FID rising from 50.99 to 59.65 on Regular.

Limitations

• Dataset may inherit MLLM biases and annotation noise from Qwen3-VL-32B captioning and grounding.
• May underrepresent unusual interactions, transparent objects, and rare categories.
• Local caption length benefits are confounded with object count and dataset composition in ablations.

Experimental setup

Datasets: LAION-Aesthetics V2 (4.75 subset) OverLay++ EliGen dataset LayoutSAM GRIT

Benchmarks: LayoutSAM-Eval OverLayBench DenseLayout

Metrics: Spatial Color Texture Shape FID IS mIoU O-mIoU SR_E SR_R

Baselines: GLIGEN Ranni MIGC InstanceDiffusion Be Yourself HiCo 3DIS SiamLayout-SD3 EliGen-SD3 EliGen-FLUX

Builds on: EliGen SiamLayout LAION-Aesthetics V2 Qwen3-VL

Closest papers by meaning

• OverLayBench: A Benchmark for Layout-to-Image Generation with Dense Overlaps (NeurIPS 2025 · similarity 0.93)
• Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation (NeurIPS 2024 · similarity 0.88)
• Learning to Generate Multiple Objects from Dense and Occluded Layouts (NeurIPS 2026 · similarity 0.88)
• InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attention (NeurIPS 2025 · similarity 0.87)
• Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation (ICLR 2025 · similarity 0.87)
• HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation (NeurIPS 2024 · similarity 0.86)