Research map / Image generation / Editing and style transfer
HIVE: Harnessing Human Feedback for Instructional Visual Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HIVE, which collects human rankings of instructional image-editing outputs, trains a BLIP-based reward model, and fine-tunes Stable Diffusion with reward-weighted or reward-conditioned losses, alongside a new 1.1M-triplet training set.
Problem
Instructional image editing models (e.g., InstructPix2Pix) often fail to follow editing instructions precisely or make unwanted changes to uninstructed regions, and standard RLHF (PPO) is costly and hard to apply to diffusion models.
Approach
Learn a reward model from human rankings of edited images and fine-tune the diffusion model offline either by exponential reward-weighted loss or by conditioning on a quantized reward text prompt, avoiding on-policy RL through the denoising chain.
Key results
- On the synthetic benchmark, HIVE gives better image consistency than IP2P-Ours at equal directional similarity.
- In user studies on the 1K real-image set, HIVE receives about 25% more votes than IP2P-Ours.
- IP2P-Ours gets about 30% more votes than the official InstructPix2Pix model, showing the benefit of the larger dataset alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add human-feedback reward fine-tuning | IP2P-Ours · Synthetic evaluation dataset | CLIP image similarity at a given directional CLIP similarity | Helped |
| Add human-feedback reward fine-tuning | IP2P-Ours · 1K real-image evaluation dataset | human preference votes: around 25% more favorites | Helped |
| Use our training data | InstructPix2Pix · 1K real-image evaluation dataset | human preference votes: around 30% more votes | Helped |
| Add cycle-consistency augmentation (ablation) | HIVE · Synthetic evaluation dataset and 1K evaluation dataset | human preference | Helped |
| Replace condition reward loss with weighted reward loss (ablation) | HIVE · 1K evaluation dataset | human preference | No clear effect |
| Upgrade the backbone from Stable Diffusion v1.5 to v2.1 (ablation) | HIVE · 1K evaluation dataset | human preference | Helped |
| Condition on the reward scalar instead of the reward prompt (ablation) | HIVE · 1K evaluation dataset | human preference votes: 25.8 % more votes | Hurt |
| Add human-feedback reward fine-tuning | IP2P-Ours · Top five verbs: add | success rate: 23.5% -> 28.7% | Helped |
| Use weighted reward loss instead of condition reward loss (ablation) | HIVE · 1K evaluation dataset | human preference | No clear effect |
Surprising findings
- Directly conditioning on the highest reward scalar underperforms the quantized text-prompt reward condition (HIVE obtains 25.8% more votes), attributed to overfitting.
- Weighted reward loss and conditional reward loss yield similar human preferences, with the conditional variant only slightly better.
Limitations
- Cannot improve cases where all model outputs are the same wrong result, so human preference offers no gain.
- Fails on counting, spatial reasoning, and instructions like 'zoom in'.
- Sometimes produces unstructured changes compared with Prompt-to-Prompt; inherits biases from Stable Diffusion, GPT-3, and BLIP.
Experimental setup
Datasets: InstructPix2Pix training data Laion-Aesthetics V2 3.6K reward fine-tuning dataset 1K human evaluation dataset
Benchmarks: InstructPix2Pix synthetic evaluation dataset (15,652 pairs) 1K real-image user study dataset
Metrics: CLIP image similarity directional CLIP similarity user study majority vote win rate success rate on top five verbs
Baselines: InstructPix2Pix official model (IP2P-Official) IP2P-Ours HIVE-Reward
Builds on: InstructPix2Pix Stable Diffusion Prompt-to-Prompt BLIP InstructGPT
Closest papers by meaning
- AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea (CVPR 2025 · similarity 0.88)
- Multi-Reward as Condition for Instruction-based Image Editing (ICLR 2025 · similarity 0.87)
- UltraEdit: Instruction-based Fine-Grained Image Editing at Scale (NeurIPS 2024 · similarity 0.87)
- Multimodal Large Language Models Make Text-to-Image Generative Models Align Better (NeurIPS 2024 · similarity 0.87)
- Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization (NeurIPS 2025 · similarity 0.87)
- InstructVideo: Instructing Video Diffusion Models with Human Feedback (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.