Research map / Large language models / Language, safety and interpretability
RetouchGPT: LLM-based Interactive High-Fidelity Face Retouching via Imperfection Prompting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RetouchGPT, an LLM-guided interactive face retouching framework that predicts imperfection masks from user instructions and modifies imperfection features via prompt-conditioned cross-attention.
Problem
Existing face retouching methods are non-interactive end-to-end image-to-image translators that cannot accept or respond to user instructions for removing specific facial imperfections while preserving attributes.
Approach
Combines an instruction-driven imperfection prediction module (text and image features yield a mask), an LLM-based embedding module that turns multimodal conditioning into imperfection prompts used to modulate a latent retouching transformer's cross-attention, and a reinforcement-style progressive retouching strategy.
Key results
- RetouchGPT reaches 46.21 PSNR and 0.0023 LPIPS on FFHQR, beating RetouchFormer (45.66, 0.0071).
- On FFHQR-hard RetouchGPT achieves 39.68 PSNR vs 38.94 for RetouchFormer.
- User/ChatGPT-4o interactive rating of 8.73 vs 5.93 for the best competing method RetouchFormer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RetouchGPT instead of RetouchFormer | face retouching · FFHQR | PSNR: 45.66 -> 46.21 | Helped |
| use RetouchGPT instead of RetouchFormer | face retouching · FFHQR | LPIPS: 0.0071 -> 0.0023 | Helped |
| use RetouchGPT instead of RetouchFormer | face retouching · FFHQR-hard | PSNR: 38.94 -> 39.68 | Helped |
| use RetouchGPT instead of RetouchFormer | face retouching · FFHQR-hard | LPIPS: 0.0157 -> 0.0115 | Helped |
| use RetouchGPT instead of VRetouchEr | face video retouching · FFHQR-seq test set | PSNR: 39.55 -> 40.12 | Helped |
| use RetouchGPT instead of VRetouchEr | face video retouching · FFHQR-seq test set | VFID: 10.01 -> 7.01 | Helped |
| add text encoder Etxt to the imperfection prediction module (ablation) | RetouchGPT · Imperfection prediction | ROUGE-1: 37.02 -> 89.72 | Helped |
| add IIP and LBE (ablation) | RetouchGPT · Interactive retouching | User rating: 4.28 ± 1.84 -> 8.73 ± 0.45 | Helped |
| add IIP (ablation) | RetouchGPT · Interactive retouching | User rating: 5.67 ± 1.42 -> 8.73 ± 0.45 | Helped |
| add LBE (ablation) | RetouchGPT · Interactive retouching | ChatGPT4-o rating: 6.87 ± 0.43 -> 8.11 ± 0.35 | Helped |
Limitations
- Relies on ChatGPT-4o for labeling imperfection categories, positions and instructions, so annotations are model-generated.
- Reported comparisons use a single generic instruction for baselines and subjective ratings from a user study and ChatGPT-4o judge.
- Generalization to video is shown only qualitatively and with a quantitative test on one FFHQR-seq protocol without video-specific training.
Experimental setup
Datasets: FFHQ-Retouching (FFHQR) FFHQR-hard (1.3k collected test samples) 1k in-the-wild images
Benchmarks: FFHQR FFHQR-hard FFHQR-seq (face video)
Metrics: PSNR SSIM LPIPS VFID Soft-IoU ROUGE-1 user rating
Baselines: AutoRetouch BPFRe ControlNet DiffBIR RetouchFormer AnomalyGPT VRetouchEr
Builds on: RetouchFormer VRetouchEr BPFRe AnomalyGPT LLaVA
Closest papers by meaning
- RetouchFormer: Semi-supervised High-Quality Face Retouching Transformer with Prior-Based Selective Self-Attention (AAAI 2024 · similarity 0.91)
- VRetouchEr: Learning Cross-frame Feature Interdependence with Imperfection Flow for Face Retouching in Videos (CVPR 2024 · similarity 0.89)
- CA-Edit: Causality-Aware Condition Adapter for High-Fidelity Local Facial Attribute Editing (AAAI 2025 · similarity 0.85)
- DiffRetouch: Using Diffusion to Retouch on the Shoulder of Experts (AAAI 2025 · similarity 0.85)
- RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward (CVPR 2026 · similarity 0.84)
- JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.