Research map / Image generation / Text-to-image diffusion
GenSpace: Benchmarking Spatially-Aware Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GenSpace, a benchmark of 1,800 text-to-image and 1,800 image-editing spatial prompts across 9 sub-domains, plus a 3D-reconstruction-based evaluation pipeline that aligns better with humans than VLM judges.
Problem
Current image generators are not evaluated on fine-grained 3D spatial awareness (object pose, relations, metric measurements), and general VLM judges fail to reliably detect such spatial errors.
Approach
A spatial expertise pipeline chains visual foundation models (Grounded-SAM, Depth Anything, OrientAnything, WildCamera, PerspectiveFields) to reconstruct 3D object and camera states from a generated image, then scores deviations from the target spatial state with a multiplicative, sub-domain-specific metric.
Key results
- The proposed evaluation pipeline reaches 76.22% average agreement with human labels, versus 56.44% for the best VLM judge (Gemini-2.5-Pro).
- GPT-4o is the top text-to-image model with average rank 1.8, scoring 94.55 on Egocentric relations and 25.01 on Complex Pose, while most sub-domains remain far from solved.
- Specialized editing models such as InstructPix2Pix, ICEdit and Step1X-Edit score below 10 on most spatial pose and measurement sub-domains, while unified models perform better.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the specialized spatial evaluation pipeline and metric instead of a general-purpose VLM evaluator | Spatial evaluation on manually labeled images · Human alignment | average agreement accuracy: 56.44 -> 76.22 | Helped |
| Use GPT-4o rather than specialized image generation models | Text-to-image generation · GenSpace text-to-image benchmark | average rank: 4.0 -> 1.8 | Helped |
| Use GPT-4o rather than specialized image editing models | Instruction-based image editing · GenSpace image editing benchmark | average rank: 2.7 -> 1.8 | Helped |
| Specify different quantitative spatial measurements in prompts | Current image generation models · GenSpace spatial measurement | spatial measurement score | No clear effect |
| Describe the desired object orientation directly rather than indirectly through camera viewpoint | Current image generation models · GenSpace camera and object pose | pose accuracy | Helped |
Surprising findings
- Models largely fail at allocentric transformation, often reversing left/right, and almost never generate specified metric distances such as 2m or 4m.
- GPT-o3 scores lower than the proposed pipeline by a wide margin in human agreement, and general VLM judges struggle most on allocentric and measurement sub-domains.
Limitations
- Evaluation pipeline depends on the accuracy of several visual foundation models, which may introduce errors.
- Benchmark coverage is limited to 50 object categories and fixed templates, and results will change as new models are added.
- Object size is evaluated mainly through relative sizes because absolute size from a monocular image is ambiguous.
Experimental setup
Benchmarks: GenSpace
Metrics: orientation difference score relation correctness relative distance error score human agreement accuracy
Baselines: Gemini-2.5-Pro GPT-4o GPT-o3 qwen-vl-max claude-3-7-sonnet-thinking SD-1.5 SD-XL DALL-E 3 SD-3.5-L FLUX.1-dev Seedream-3.0 Bagel
Builds on: GenEval T2I-CompBench Grounded-SAM Depth Anything OrientAnything
Closest papers by meaning
- Exploring Spatial Intelligence from a Generative Perspective (CVPR 2026 · similarity 0.89)
- Let Language Constrain Geometry: Vision–Language Models as Semantic and Spatial Critics for 3D Generation (ICML 2026 · similarity 0.88)
- InfiniBench: Infinite Benchmarking for Visual Spatial Reasoning with Customizable Scene Complexity (CVPR 2026 · similarity 0.87)
- SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent (NeurIPS 2025 · similarity 0.87)
- Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation (ICLR 2026 · similarity 0.87)
- A 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.