Research map / Image generation / Text-to-image diffusion
MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humans
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MultiHuman-Testbench, a 1,800-sample benchmark with 5,550 face IDs and four metrics for multi-human image generation, and proposes training-free regional isolation and Hungarian implicit assignment to improve OmniGen and IR-Diffusion.
Problem
Generating images with multiple specific humans performing described actions while preserving each person's facial identity and correct count lacks a dedicated benchmark and standardized metrics.
Approach
Combines a curated multi-human prompt and face-ID benchmark with Hungarian-matched ID similarity, face-count accuracy, HPS, and MLLM action QA; and adds training-free unified regional attention isolation plus implicit region assignment (attention maps + SAM + Hungarian matching) to multi-human unified models and IR-Diffusion.
Key results
- MH-OmniGen reaches a Unified score of 61.6 on Task 1 versus 59.2 for OmniGen, with Multi-ID rising from 49.4 to 54.5.
- GPT-Image-1 achieves the highest Count accuracy (87.9) and action scores in Task 1 but has Multi-ID of 28.8, about 25.7 points below MH-OmniGen.
- MH-IR-Diffusion attains the highest Count (62.6) and Multi-ID (33.3) in Task 3 among the reference-free story methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add unified regional isolation and implicit region assignment | OmniGen (MH-OmniGen) · Task 1: MultiHuman-Testbench | Multi-ID: 49.4 -> 54.5 | Helped |
| Add unified regional isolation and implicit region assignment | OmniGen (MH-OmniGen) · Task 1: MultiHuman-Testbench | Action-S: 87.5 -> 91.6 | Helped |
| Add unified regional isolation and implicit region assignment | OmniGen (MH-OmniGen) · Task 1: MultiHuman-Testbench | Unified: 59.2 -> 61.6 | Helped |
| Add unified regional isolation and implicit region assignment | OmniGen (MH-OmniGen) · Task 1: MultiHuman-Testbench | Count: 60.5 -> 60.3 | Hurt |
| Add implicit face-region matching and regional isolation | IR-Diffusion (MH-IR-Diffusion) · Task 3: MultiHuman-Testbench without reference images | Multi-ID: 27.6 -> 33.3 | Helped |
| Add implicit face-region matching and regional isolation | IR-Diffusion (MH-IR-Diffusion) · Task 3: MultiHuman-Testbench without reference images | Action-C: 71.8 -> 71.3 | Hurt |
| Use pose regional priors | OmniGen · MultiHuman-Testbench reference-based generation | Count: 60.5 -> 77.2 | Helped |
| Use pose regional priors | OmniGen · MultiHuman-Testbench reference-based generation | Multi-ID: 49.4 -> 48.2 | No clear effect |
| Train using 5 views instead of 1 view | LoRA · Task 1: MultiHuman-Testbench | Multi-ID: 20.2 -> 22.0 | Helped |
| Train using 5 views instead of 1 view | MuDI · Task 1: MultiHuman-Testbench | Multi-ID: 23.6 -> 24.6 | Helped |
Surprising findings
- GPT-Image-1 achieves the best count accuracy and action scores yet poor identity preservation, attributed to hallucinated facial features.
- No evaluated method, with or without regional priors, consistently achieves both correct human count and high identity similarity.
Limitations
- None of the evaluated methods consistently achieves high count accuracy and identity similarity together.
- Pose priors for Task 2 were partly human-selected and generated via text-to-pose, with manual filtering required.
- Dataset labels for age, ethnicity, and gender were produced by Gemini Flash 2.0 rather than manual annotation.
Experimental setup
Datasets: FFHQ SFHQ CelebA-HQ
Benchmarks: MultiHuman-Testbench
Metrics: Count Accuracy Hungarian ID similarity HPS Action-S Action-C Unified score
Baselines: GPT-Image-1 LoRA MuDI IP-Adapter MIP-Adapter Lambda-Eclipse UniPortrait RectifID FastComposer OmniGen Flux-Kontext Regional-PuLID
Builds on: OmniGen IR-Diffusion MuDI Show-o ArcFace
Closest papers by meaning
- CapHuman: Capture Your Moments in Parallel Universes (CVPR 2024 · similarity 0.88)
- OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation (CVPR 2025 · similarity 0.87)
- Video-Bench: Human-Aligned Video Generation Benchmark (CVPR 2025 · similarity 0.87)
- AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmark (CVPR 2025 · similarity 0.87)
- DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation (ICLR 2025 · similarity 0.87)
- HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.