Research map / Image generation / Text-to-image diffusion
GenScale: A Benchmark for Relative Object Scale in Image Generation and Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GenScale, a benchmark of 900 image-level entries (1,643 pairwise anchor-target scale relations) for relative object scale in image generation and editing, with a human-calibrated Gemini ordinal judge and Rescale, a training-free agentic post-hoc scale-correction pipeline.
Problem
Image generation and editing systems often render familiar objects at implausible relative sizes (mean regression toward typical scale), and existing benchmarks do not evaluate relative object scale as a distinct capability.
Approach
Represent scale as pairwise anchor-target relations grounded in external physical-size metadata (COCO/LVIS categories, ABO products, anthropometric data), score them with a five-point ordinal rubric calibrated against nine human raters using a Gemini judge, and correct errors with a model-agnostic agent that segments, removes, and reinserts objects at a planned resize factor.
Key results
- Best Task 1 model Nano Banana 2 reaches only 56.5% plausible relations on S1+S2 common-object generation.
- Gemini judge vs. human consensus achieves QWK 0.823 and within-one agreement 97.15% on T1+T2.
- Rescale reduces Task 3 scale error from 1.258 to 0.548 (gain +0.710), versus at most +0.601 for GPT-Image-2 in S4+S5 editing.
Surprising findings
- Natural-depth scenes (S1) hide scale errors: generators place smaller objects closer and larger ones farther, so S1 plausibility is much higher than same-plane S2.
- Human-product generations show asymmetric product magnification, with products more often enlarged than shrunk.
Limitations
- Covers only visually identifiable categories with stable physical dimensions; excludes deformable or context-dependent objects.
- Rescale assumes scale-relevant objects are visible and locally editable.
- SD3.5-Large has much lower scored coverage on Task 3, limiting comparability; Gemini judge is itself a proprietary VLM.
Experimental setup
Datasets: COCO LVIS Amazon Berkeley Objects
Benchmarks: GenScale
Metrics: scale error (mean |s-3|) plausible rate mean-regression error severe rate CLIP-I DINO SSIM SSIM-HF LAION-Aes Q-Align-IQ
Baselines: Nano Banana 2 GPT-Image-2 Z-Image-Turbo Grok Imagine Qwen-Image 2512 FLUX.2 SD3.5-Large Seedream v4.5 Qwen-Image-Edit-2511 FLUX.1 Kontext-dev
Builds on: GenSpace SAM 2 Depth Anything V2 InsertAnything COCO LVIS Amazon Berkeley Objects
Closest papers by meaning
- UltraEdit: Instruction-based Fine-Grained Image Editing at Scale (NeurIPS 2024 · similarity 0.87)
- GenSpace: Benchmarking Spatially-Aware Image Generation (NeurIPS 2025 · similarity 0.86)
- From Scale to Speed: Adaptive Test-Time Scaling for Image Editing (CVPR 2026 · similarity 0.86)
- ImgEdit: A Unified Image Editing Dataset and Benchmark (NeurIPS 2025 · similarity 0.86)
- MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (NeurIPS 2025 · similarity 0.86)
- ImagenHub: Standardizing the evaluation of conditional image generation models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.