Research map / Image generation / Text-to-image diffusion
Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes image regeneration as a text-to-image evaluation task, where a T2I model regenerates a reference image via MLLM-guided prompt construction (IUT) and iterative revision, then compares generated and reference images.
Problem
Text-to-image models are evaluated by matching input text to output images, which suffers from cross-modal information asymmetry and yields unreliable or incomplete assessments of overall generative quality.
Approach
Reframe T2I evaluation as image-to-image regeneration: an MLLM (GPT-4V) converts a reference image into a text prompt via a hierarchical Image Understanding Tree, and an iterative loop (prompt generation/revision, image generation, selection by CLIP/DINOv2/GPT-4V similarity, feedback) produces a regenerated image compared against the reference.
Key results
- Juggernautv9 scores highest across metrics (CLIP 93.79%, DINO 95.34%, GPT4-con 0.77) and user study consistency 0.7056 among evaluated SD variants.
- IUT prompt generation improves JuggernautXL v9 GPT4v score from 76.8% to 84.4% over direct prompts.
- Direct GPT-4V text-image matching fails to distinguish models well compared with image regeneration (e.g., SD1.5 0.6620 vs Juggernautv9 0.7220 direct, against 0.5210 vs 0.8260 under regeneration).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use image regeneration instead of direct GPT4v text-image matching (ablation) | ImageRepainter · Content-diverse benchmark, JuggernautXL v9 | GPT4v: 0.7220 -> 0.8260 | Helped |
| Use image regeneration instead of direct GPT4v text-image matching (ablation) | ImageRepainter · Content-diverse benchmark, SD1.5 | GPT4v: 0.6620 -> 0.5210 | Hurt |
| Use image regeneration instead of direct GPT4v text-image matching (ablation) | ImageRepainter · Content-diverse benchmark, JuggernautXL v1 | GPT4v: 0.6960 -> 0.7430 | Helped |
| Add IUT for initial prompt generation (ablation) | ImageRepainter with JuggernautXL v9 · Content-diverse benchmark | CLIP: 93.66 -> 95.71 (+2.05) | Helped |
| Add IUT for initial prompt generation (ablation) | ImageRepainter with JuggernautXL v9 · Content-diverse benchmark | DINO: 94.83 -> 95.71 (+0.88) | Helped |
| Add IUT for initial prompt generation (ablation) | ImageRepainter with JuggernautXL v9 · Content-diverse benchmark | GPT4v: 76.8 -> 84.4 (+7.6) | Helped |
| Add IUT for initial prompt generation (ablation) | ImageRepainter with SDXL1.0 · Content-diverse benchmark | CLIP: 90.18 -> 90.17 (-0.01) | Hurt |
| Add IUT for initial prompt generation (ablation) | ImageRepainter with SDXL1.0 · Content-diverse benchmark | DINO: 90.41 -> 91.48 (+1.07) | Helped |
| Add IUT for initial prompt generation (ablation) | ImageRepainter with SDXL1.0 · Content-diverse benchmark | GPT4v: 62.8 -> 66.0 (+3.2) | Helped |
| Use more iteration rounds (ablation) | ImageRepainter with SDXL1.0 · Image regeneration | image quality | Helped |
| Use more iteration rounds (ablation) | ImageRepainter with JuggernautXL v9 · Image regeneration | image quality | No clear effect |
Surprising findings
- Direct GPT-4V text-image matching gives SDXL1.0 the highest score (0.7440), above Juggernautv9, contrary to human judgments.
- On T2I-CompBench, SDXL1.0 ranks highest for content consistency though it diverges from human judgments.
Limitations
- Evaluation relies on GPT-4V as the MLLM and as a judge, which may introduce bias and is closed-source.
- Small benchmarks (100 and 200 samples) and a 40-annotator user study limit statistical strength.
- Content such as portraits and multi-object images remains poorly generated even after iterations.
Experimental setup
Datasets: content-diverse benchmark (100 samples, 4 content types) style-diverse benchmark (200 samples, 10 styles)
Benchmarks: Image Regeneration content-diverse benchmark Image Regeneration style-diverse benchmark T2I-CompBench
Metrics: CLIP similarity DINOv2 similarity GPT-4V content consistency score GPT-4V perceptual quality score user study consistency and perceptual scores HPSv2 PickScore
Baselines: T2I-CompBench HPSv2 PickScore CLIP-interrogator Direct GPT-4V text-image matching
Builds on: T2I-CompBench CLIP-interrogator ImagenHub
Closest papers by meaning
- Exploring the Deep Fusion of Large Language Models and Diffusion Transformers for Text-to-Image Synthesis (CVPR 2025 · similarity 0.91)
- Rich Human Feedback for Text-to-Image Generation (CVPR 2024 · similarity 0.91)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.90)
- Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.90)
- Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.90)
- Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.