Research map / Image generation / Text-to-image diffusion
Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| finetune CLIP-H on real-user preference pairs (PickScore) | CLIP-H · Pick-a-Pic test | accuracy: 60.8 -> 70.5 | Helped |
| use PickScore instead of FID for model ranking | T2I model evaluation · MS-COCO val | correlation with human: -0.900 -> 0.917 | Helped |
| use PickScore instead of HPS as evaluation metric | Elo model ranking · Pick-a-Pic test | correlation with real users: 0.670 -> 0.790 | Helped |
| select best of 100 images with PickScore vs CLIP-H | Dreamlike Photoreal 2.0 · Pick-a-Pic prompts | human win rate: 71.3 | Helped |
| add in-batch negatives to preference objective (ablation) | PickScore · Pick-a-Pic test | accuracy: 70.5 -> 65.2 | Hurt |
Closest papers by meaning
- Learning Multi-Dimensional Human Preference for Text-to-Image Generation (CVPR 2024 · similarity 0.90)
- Multimodal Large Language Models Make Text-to-Image Generative Models Align Better (NeurIPS 2024 · similarity 0.87)
- Revisiting text-to-image evaluation with Gecko: on metrics, prompts, and human rating (ICLR 2025 · similarity 0.87)
- A User-Friendly Framework for Generating Model-Preferred Prompts in Text-to-Image Synthesis (AAAI 2024 · similarity 0.87)
- Preference Adaptive and Sequential Text-to-Image Generation (ICML 2025 · similarity 0.86)
- SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.