Research map / Image generation / Text-to-image diffusion
Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Community
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Social Reward, a CLIP-based reward model trained on a million-user implicit feedback dataset (Picsart Image-Social) from remix behavior on an online creative platform, and uses it to fine-tune Stable Diffusion 2.1-base.
Problem
Existing text-to-image reward models are trained on small, professionally annotated preference data focused on fidelity and alignment, and do not capture community-level creative preference for popular AI-generated art on social platforms.
Approach
Use implicit social feedback (how often generated images are remixed by community members, plus influencer remixes) as preference labels at million-user scale, train a CLIP-based triplet-loss scorer on prompt, positive, and negative image triplets, and use it to fine-tune a diffusion model toward popular creative outputs.
Key results
- Social Reward reaches 69.7% pairwise accuracy on the Picsart Image-Social test set, above PickScore (62.6%), ImageReward (60.48%) and HPS v2 (59.4%).
- CLIP alignment and LAION aesthetic scores achieve only 51.9% and 55.3% accuracy on the Picsart Image-Social test set, showing existing general metrics poorly capture community preference.
- Fine-tuning Stable Diffusion 2.1-base with Social Reward improves Social Reward win rate to 75.6% against the baseline, and also improves LAION aesthetic and HPS v2 scores, while CLIP alignment slightly decreases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train CLIP on Picsart social-feedback preferences | Social Reward · Picsart Image-Social test | pairwise accuracy: 62.6% -> 69.7% | Helped |
| Fine-tune Stable Diffusion 2.1-base using Social Reward | Stable Diffusion 2.1-base · general prompts | Social Reward: -0.095 -> -0.062 | Helped |
| Fine-tune Stable Diffusion 2.1-base using Social Reward | Stable Diffusion 2.1-base · internal prompts | Social Reward: -0.130 -> -0.093 | Helped |
| Fine-tune Stable Diffusion 2.1-base using Social Reward | Stable Diffusion 2.1-base · general prompts | CLIP alignment: 0.280 -> 0.278 | Hurt |
| Fine-tune Stable Diffusion 2.1-base using Social Reward | Stable Diffusion 2.1-base · internal prompts | CLIP alignment: 0.260 -> 0.256 | Hurt |
| Use Triplet loss instead of InfoNCE (ablation) | Social Reward · Picsart Image-Social test | pairwise accuracy: 68.5% -> 69.7% | Helped |
| Use Triplet loss instead of Contrastive loss (ablation) | Social Reward · Picsart Image-Social test | pairwise accuracy: 67.1% -> 69.7% | Helped |
| Use Triplet loss instead of BCE (ablation) | Social Reward · Picsart Image-Social test | pairwise accuracy: 67% -> 69.7% | Helped |
| Use Triplet loss instead of Reweighted Cross-Entropy (ablation) | Social Reward · Picsart Image-Social test | pairwise accuracy: 65.9% -> 69.7% | Helped |
Surprising findings
- Vanilla CLIP alignment and LAION aesthetic scores are nearly at chance on the Picsart social preference data, suggesting community preference is distinct from fidelity and alignment.
- Fine-tuning for social popularity slightly lowered CLIP text-image alignment, which the authors attribute to users favoring editable images over prompt fidelity.
Limitations
- Dataset is proprietary and tied to one platform (Picsart), limiting generalizability and reproducibility.
- Implicit remix signals are noisy and biased by exposure and Pareto-like engagement; the user study is small (100 prompts, creators only).
- Fine-tuning evaluation uses only 4 images per prompt and a single base model, with limited baselines.
Experimental setup
Datasets: Picsart Image-Social LAION-5B subset DiffusionDB HPD v2 ImageReward dataset Pick-a-Pic
Benchmarks: Picsart Image-Social test set DiffusionDB prompts (general) internal validation prompts user study with Picsart creators
Metrics: pairwise accuracy KL divergence win rate average scores (Social Reward, CLIP alignment, LAION aesthet
Baselines: PickScore ImageReward HPS v2 CLIP alignment score LAION aesthetic score
Builds on: CLIP ImageReward PickScore Stable Diffusion 2.1 LAION-5B
Closest papers by meaning
- ArtWhisperer: A Dataset for Characterizing Human-AI Interactions in Artistic Creations (ICML 2024 · similarity 0.88)
- Stylebreeder: Exploring and Democratizing Artistic Styles through Text-to-Image Models (NeurIPS 2024 · similarity 0.87)
- HIVE: Harnessing Human Feedback for Instructional Visual Editing (CVPR 2024 · similarity 0.86)
- PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference (NeurIPS 2024 · similarity 0.86)
- An analytic theory of creativity in convolutional diffusion models (ICML 2025 · similarity 0.85)
- Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist Rewards (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.