Research map / Image generation / Text-to-image diffusion

Make It Count: Text-to-Image Generation with an Accurate Number of Objects

CVPR 2025 · Image generation › Text-to-image diffusion · approach: Text-driven synthesis and semantic guidance · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CountGen, which finds instance-identity features in SDXL self-attention, corrects object counts in the intermediate layout with a trained ReLayout U-Net, and guides denoising with layout losses to generate the prompted number of objects.

Problem

Text-to-image diffusion models often generate the wrong number of objects specified in the prompt, especially for counts above three.

Approach

Uses SDXL self-attention features (layer up_52 at t=500) with cross-attention foreground masks and DBSCAN to detect instances mid-denoising; a U-Net (ReLayout) trained on self-generated k vs k+1 mask pairs adds or removes instances; then test-time cross-attention layout loss and self-attention masking enforce the corrected layout.

Key results

• CountGen raises count accuracy on CoCoCount from 26% (SDXL) to 52% by human rating and 50% by YOLOv9.
• On T2I-CompBench-Count, human count accuracy rises from 29% (SDXL) to 48%, above DALL-E 3's 36%.
• In a user study, SDXL was preferred over CountGen in only 23 of 200 image-quality comparisons.

Measured effects reported in the paper

Surprising findings

• Self-attention features from one mid-network up-block layer (up_52 at t=500) separate object instances without supervision, though most layers do not.
• Removing self-attention masking lowers precision, showing that the layout loss alone causes objects to appear in the background.

Limitations

• Occasionally multiple instances appear in a region meant for one object, and backgrounds become plainer than SDXL.
• Evaluation limited to single-object prompts with counts from 2 to 10.
• Over-generation mostly occurs for counts above 5, and layout-guidance issues for counts 5 or fewer.

Experimental setup

Datasets: T2I-CompBench-Count (218 prompts) self-generated SDXL k/k+1 mask pairs

Benchmarks: T2I-CompBench-Count CoCoCount

Metrics: count accuracy (human) count accuracy (YOLOv9 detection) precision recall IOU image quality preference

Baselines: SDXL Repeated Object Reason Out Your Layout DALL-E 3 Random masks + Bounded Attention Counting Guidance RPG Token Optimization

Builds on: SDXL Bounded Attention Attend-and-Excite Layout-guided generation DBSCAN

Closest papers by meaning

• T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting (CVPR 2025 · similarity 0.88)
• RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.88)
• The Crystal Ball Hypothesis in diffusion models: Anticipating object positions from initial noise (ICLR 2025 · similarity 0.87)
• Clockwork Diffusion: Efficient Generation With Model-Step Distillation (CVPR 2024 · similarity 0.87)
• Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling (NeurIPS 2024 · similarity 0.87)
• LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.87)