Research map / Image generation / Editing and style transfer
AutoEdit: Automatic Hyperparameter Tuning for Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Casts per-image hyperparameter selection for diffusion-based image editing as an MDP and trains a PPO policy that picks hyperparameters at each denoising step, reducing brute-force search cost.
Problem
Diffusion-based text-guided image editing methods require brute-force tuning of multiple interdependent hyperparameters (e.g., inversion timestep, attention replacement ratios) per image, which is computationally expensive.
Approach
Treat each denoising step as an RL state and the set of editing hyperparameters as per-step actions chosen by a PPO policy (initialized from a prior-sampled pretraining phase), with a reward combining CLIP-based edit alignment and background MSE preservation, so hyperparameter search runs in a single denoising trajectory.
Key results
- Adding AutoEdit to DDPM-Inversion (SD 1.4) raises PSNR from 22.66 to 27.25 and lowers MSE from 53.33 to 31.18 with only a small CLIP edited-score drop (23.02 to 22.52).
- AutoEdit adds about 0.6% inference time and parameters (86M to 87.1M) to the SD U-Net.
- In a user study with 50 participants, 82.78% of responses preferred AutoEdit outputs over baseline hyperparameter settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply AutoEdit hyperparameter tuning | DDIM-Inversion · PieBench | PSNR: 21.36 -> 24.65 | Helped |
| apply AutoEdit hyperparameter tuning | DDIM-Inversion · PieBench | CLIP Edited: 23.30 -> 22.65 | Hurt |
| apply AutoEdit hyperparameter tuning | DDPM-Inversion · PieBench | PSNR: 22.66 -> 27.25 | Helped |
| apply AutoEdit hyperparameter tuning | DDPM-Inversion · PieBench | CLIP Edited: 23.02 -> 22.52 | Hurt |
| apply AutoEdit hyperparameter tuning | PnP Inversion · PieBench | CLIP Edited: 22.10 -> 23.00 | Helped |
| apply AutoEdit hyperparameter tuning | P2P · PieBench | CLIP Edited: 22.29 -> 23.88 | Helped |
| remove Phase 1 policy initialization (ablation) | AutoEdit with Phase 1 and Phase 2 · DDPM-Inversion editing | PSNR: 27.2 -> 18.2 | Hurt |
| use Phase 1 without Phase 2 (ablation) | AutoEdit with Phase 1 and Phase 2 · DDPM-Inversion editing | PSNR: 27.2 -> 22.1 | Hurt |
| increase reward background coefficient β from 10 to 40 (ablation) | AutoEdit · DDPM-Inversion editing | PSNR: 19.65 -> 28.53 | Helped |
| increase reward background coefficient β from 10 to 40 (ablation) | AutoEdit · DDPM-Inversion editing | CLIP Edited: 24.15 -> 21.36 | Hurt |
| replace CLIP-based reward with LLM-based reward (ablation) | AutoEdit · DDPM-Inversion editing | LLM Score: 1.27 -> 1.31 | Helped |
Surprising findings
- The CLIP score is reported as insensitive for judging edit quality, so the authors accept a small CLIP drop; CLIP-based reward fails on small-region edits such as eye color.
Limitations
- Performance is bounded by the underlying editing method; if the base method fails on an image, AutoEdit inherits the failure.
- Reported CLIP-based reward is weak for small-region edits, requiring an LVLM reward instead.
- Evaluation relies on PieBench and user study of 50 participants, with a limited set of hyperparameters per method.
Experimental setup
Datasets: EditBench PieBench
Benchmarks: PieBench
Metrics: Structure Distance PSNR SSIM MSE LPIPS CLIP score (edited region) CLIP score (whole image) LLM Score inference time user preference
Baselines: DDIM-Inversion DDPM-Inversion PnP Inversion Prompt-to-Prompt MasaCtrl Null-text inversion InstructPix2Pix UltraEdit OIR Taming Flow Fireflow
Builds on: PPO prompt-to-prompt DDIM inversion DDPM inversion PieBench
Closest papers by meaning
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.90)
- Inversion-Free Image Editing with Language-Guided Diffusion Models (CVPR 2024 · similarity 0.88)
- h-Edit: Effective and Flexible Diffusion-Based Editing via Doob's h-Transform (CVPR 2025 · similarity 0.88)
- FeedEdit: Text-Based Image Editing with Dynamic Feedback Regulation (CVPR 2025 · similarity 0.88)
- Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer (NeurIPS 2025 · similarity 0.88)
- Posterior Distillation Sampling (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.