Research map / 3D vision / Face and shape
FlowDrag: 3D-aware Drag-based Image Editing with Mesh-guided Deformation Vector Flow Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FlowDrag, which builds a 3D mesh from an image, deforms it with progressive SR-ARAP, and injects the projected 2D displacement field into diffusion-based drag editing, plus a new video-derived ground-truth benchmark (VFD-Bench).
Problem
Drag-based image editing with diffusion models matches only user-given handle points, which causes geometric inconsistency and artifacts on rigid parts of objects; existing benchmarks (DragBench) also lack ground-truth edited images.
Approach
Use a 3D mesh built from the image (depth-based or image-to-3D diffusion) deformed by progressive SR-ARAP to produce a dense 2D vector flow that supervises motion supervision and point tracking, plus layout feature injection from the projected deformed mesh, so the whole object moves coherently.
Key results
- On VFD-Bench, FlowDrag reaches PSNR 18.55, 1-LPIPS 0.82 and MD 28.23, beating GoodDrag (18.14, 0.79, 35.31).
- On DragBench, FlowDrag achieves the lowest MD of 22.88 (GoodDrag 22.96), but a lower 1-LPIPS (0.82) than DiffEditor and DragDiffusion (0.89).
- A user study with 25 volunteers ranked FlowDrag highest on drag accuracy and image quality against DiffEditor, FreeDrag and GoodDrag.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FlowDrag instead of DiffEditor | FlowDrag · DragBench | 1-LPIPS: 0.89 -> 0.82 | Hurt |
| use FlowDrag instead of GoodDrag | FlowDrag · DragBench | MD: 22.96 -> 22.88 | Helped |
| use FlowDrag instead of GoodDrag | FlowDrag · VFD-Bench | PSNR: 18.14 -> 18.55 | Helped |
| use FlowDrag instead of GoodDrag | FlowDrag · VFD-Bench | 1-LPIPS: 0.79 -> 0.82 | Helped |
| use FlowDrag instead of GoodDrag | FlowDrag · VFD-Bench | MD: 35.31 -> 28.23 | Helped |
| prioritize vectors with larger magnitudes instead of uniform sub-sampling (ablation) | FlowDrag vector-flow sampling · VFD-Bench | MD: 30.21 -> 28.23 | Helped |
| prioritize vectors with larger magnitudes instead of uniform sub-sampling (ablation) | FlowDrag vector-flow sampling · VFD-Bench | 1-LPIPS: 0.80 -> 0.82 | Helped |
| set Inter-Step Smoothness β to 0.8 rather than β=0.2 (ablation) | Progressive SR-ARAP · VFD-Bench mesh deformation | MELR: 0.87 -> 0.94 | Helped |
| set Inter-Step Smoothness β to 0.8 rather than β=0.2 (ablation) | Progressive SR-ARAP · VFD-Bench mesh deformation | mARAP Error: 17.24 -> 10.12 | Helped |
| use 10 sampled vectors (ablation) | FlowDrag vector-flow optimization · VFD-Bench | PSNR | Helped |
| use 10 sampled vectors (ablation) | FlowDrag vector-flow optimization · VFD-Bench | 1-LPIPS | Helped |
Surprising findings
- DiffEditor gets the highest 1-LPIPS on DragBench because it induces minimal edits, so the fidelity metric rewards weak editing.
- The paper reports that 1-LPIPS penalizes successful geometry-changing edits, which motivated the video-based GT benchmark.
Limitations
- Relies on stable mesh deformation, which limits feasible drag distances.
- Primarily supports rigid edits and struggles with content creation or removal requiring large structural changes.
- Projecting 3D deformation to 2D loses 3D information, and Stable Diffusion has no 3D understanding.
Experimental setup
Datasets: DragBench VFD-Bench
Benchmarks: DragBench VFD-Bench
Metrics: 1-LPIPS MD PSNR user study score
Baselines: DiffEditor DragDiffusion DragNoise FreeDrag GoodDrag
Builds on: DragDiffusion GoodDrag SR-ARAP DragGAN Marigold
Closest papers by meaning
- 3DGS-Drag: Dragging Gaussians for Intuitive Point-Based 3D Editing (ICLR 2025 · similarity 0.90)
- LightningDrag: Lightning Fast and Accurate Drag-based Image Editing Emerging from Videos (ICML 2025 · similarity 0.89)
- GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models (ICLR 2025 · similarity 0.89)
- CLIPDrag: Combining Text-based and Drag-based Instructions for Image Editing (ICLR 2025 · similarity 0.87)
- DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing (CVPR 2024 · similarity 0.87)
- FastDrag: Manipulate Anything in One Step (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.