Research map / Image generation / Text-to-image diffusion
MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation
Yuta Oshima, Daiki Miyake, Kohsei Matsutani, Yusuke Iwasawa, Masahiro Suzuki, Yutaka Matsuo, Hiroki Furuta
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase number of references from 3 to 8 (ablation) | all evaluated models · MultiBanana total score | total score | Hurt |
| Add iterative prompt refinement agent (IPR) (ablation) | GPT-Image-1 · MultiBanana Two references | avg score: 6.585 -> 6.808 | Improved |
| Add iterative prompt refinement agent (IPR) (ablation) | GPT-Image-1 · MultiBanana X-1 + Local | avg score: 5.147 -> 5.419 | Improved |
| Add iterative prompt refinement agent (IPR) (ablation) | Nano Banana · MultiBanana Two references | avg score: 4.891 -> 5.030 (Two); 4.118 -> 4.030 (X-1 + Local) | Mixed |
| Add iterative prompt refinement agent (IPR) (ablation) | Nano Banana · MultiBanana Single | avg score: 7.817 -> 7.606 | Hurt |
| Background replacement vs other task types (X-1 + Background) | closed and open models · MultiBanana | avg score: Nano Banana 3.575 vs 4.453 (X Objects) | Hurt |
| Cross-domain, scale/view, rare concept, multilingual references | most models · MultiBanana | score | Hurt |
Closest papers by meaning
- MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (NeurIPS 2025 · similarity 0.89)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.89)
- LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.88)
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.88)
- Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models (AAAI 2025 · similarity 0.88)
- ImagenHub: Standardizing the evaluation of conditional image generation models (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
