Research map / Video / Video segmentation and tracking
VRetouchEr: Learning Cross-frame Feature Interdependence with Imperfection Flow for Face Retouching in Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VRetouchEr, a transformer for face video retouching that estimates imperfection flow between frames to refine imperfection maps and uses multi-frame masked attention over reference frames to replace facial blemishes.
Problem
Removing facial imperfections from videos stably and precisely; image retouching methods are unstable across frames and video enhancement methods lack imperfection localization, with paired video training data scarce.
Approach
Estimate imperfection displacement flow between consecutive frames to warp and refine per-frame imperfection maps (Flow-based Imperfection Refinement), then inject these maps into a latent transformer that performs multi-frame masked cross-attention over reference frames (Multi-frame Masked Attention).
Key results
- On FFHQR-Seq, VRetouchEr reaches 39.75 PSNR, 1.06 dB above the second-best BPFRe (38.69).
- On MRFV, VRetouchEr achieves VFID 10.368 versus 18.772 for BPFRe and PSNR 37.63 versus 36.32.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VRetouchEr instead of BPFRe | Face video retouching · FFHQR-Seq | PSNR: 38.69 -> 39.75 | Helped |
| Use VRetouchEr instead of BPFRe | Face video retouching · FFHQR-Seq | VFID: 7.604 -> 6.375 | Helped |
| Use VRetouchEr instead of BPFRe | Face video retouching · MRFV | PSNR: 36.32 -> 37.63 | Helped |
| Use VRetouchEr instead of BPFRe | Face video retouching · MRFV | VFID: 18.772 -> 10.368 | Helped |
| Add FIR and MMA (ablation) | VRetouchEr base model · FFHQR-Seq | PSNR: 38.13 -> 39.75 | Helped |
| Add FIR and MMA (ablation) | VRetouchEr base model · FFHQR-Seq | VFID: 8.671 -> 6.375 | Helped |
| Add MMA (ablation) | VRetouchEr · FFHQR-Seq | PSNR: 38.67 -> 39.75 | Helped |
| Add MMA (ablation) | VRetouchEr · FFHQR-Seq | LPIPS: 0.0238 -> 0.0169 | Helped |
| Add FIR (ablation) | VRetouchEr · FFHQR-Seq | PSNR: 38.74 -> 39.75 | Helped |
| Add FIR (ablation) | VRetouchEr · FFHQR-Seq | VFID: 7.312 -> 6.375 | Helped |
| Increase MMA reference-frame count; best reported setting is δ=6 (ablation) | VRetouchEr · MRFV | VFID | Helped |
Surprising findings
- Removing the flow-based refinement (w/o FIR) yields imperfection localization comparable to single-image BPFRe, indicating the motion refinement is what drives stability gains.
Limitations
- Training uses emulated video sequences built from image pairs because paired raw-retouched videos are unavailable.
- Performance gains over the number of reference frames plateau beyond delta=6, and results depend on the chosen frame count.
Experimental setup
Datasets: FFHQR FFHQR-Seq MRFV
Benchmarks: FFHQR-Seq MRFV
Metrics: PSNR SSIM LPIPS VFID Soft-IoU user preference ranking
Baselines: Pix2PixHD AutoRetouch MPRNet GPEN RestoreFormer BIPNet ProPainter BPFRe
Builds on: SpyNet ProPainter BPFRe VGG-19
Closest papers by meaning
- RetouchGPT: LLM-based Interactive High-Fidelity Face Retouching via Imperfection Prompting (AAAI 2025 · similarity 0.89)
- RetouchFormer: Semi-supervised High-Quality Face Retouching Transformer with Prior-Based Selective Self-Attention (AAAI 2024 · similarity 0.88)
- Blemish-Aware and Progressive Face Retouching With Limited Paired Data (CVPR 2023 · similarity 0.86)
- DiffRetouch: Using Diffusion to Retouch on the Shoulder of Experts (AAAI 2025 · similarity 0.83)
- Keyframe-Guided Creative Video Inpainting (CVPR 2025 · similarity 0.83)
- Efficient Video Face Enhancement with Enhanced Spatial-Temporal Consistency (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.