Research map / Image generation / Editing and style transfer
Localize, Understand, Collaborate: Semantic-Aware Dragging via Intention Reasoner
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LucidDrag, a training-free "what-then-how" drag-based image editor that uses an LVLM (Osprey) and GPT-3.5 to reason intentions and source/target prompts, then guides diffusion sampling with semantic, image-quality (discriminator classifier guidance), and editing guidance.
Problem
Drag-based image editing is ill-posed: point-dragging methods yield one deterministic result, ignoring multiple plausible semantic intentions, and often degrade image fidelity.
Approach
Shift from "how to drag" to "what-then-how": an LVLM+LLM intention reasoner infers several semantic intentions with asymmetric source/target prompts, which feed a collaborative guidance sampler combining editing guidance, semantic prompt-direction guidance, and discriminator-based quality guidance.
Key results
- On DragBench, LucidDrag reports Mean Distance 20.46 vs 29.16 for DragonDiffusion and 26.01 for DiffEditor, and GScore 7.37 vs 7.15 for FreeDrag.
- Removing the intention reasoner raises Mean Distance to 23.66 and drops GScore to 6.76; removing quality guidance gives 22.13 and 6.47.
- On object moving, LucidDrag reports CLIP score 0.260 and LMM score 7.25, above DragonDiffusion (0.255, 6.25) and DiffEditor (0.257, 6.5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the intention reasoner (ablation) | LucidDrag · DragBench | Mean Distance: 23.66±0.73 -> 20.46±0.77 | Helped |
| add the intention reasoner (ablation) | LucidDrag · DragBench | GScore: 6.76 -> 7.37 | Helped |
| add quality guidance (ablation) | LucidDrag · DragBench | Mean Distance: 22.13±0.64 -> 20.46±0.77 | Helped |
| add quality guidance (ablation) | LucidDrag · DragBench | GScore: 6.47 -> 7.37 | Helped |
| use LucidDrag instead of the strongest baseline, DiffEditor | DragBench · DragBench | Mean Distance: 26.01±0.81 -> 20.46 ± 0.77 | Helped |
| use LucidDrag instead of the strongest baseline, FreeDrag | DragBench · DragBench | GScore: 7.15 -> 7.37 | Helped |
| use LucidDrag instead of DiffEdit | Object moving · Object moving (20 editing samples) | CLIP-score: 0.257 -> 0.260 | Helped |
| use LucidDrag instead of DiffEdit | Object moving · Object moving (20 editing samples) | LMM-score: 6.5 -> 7.25 | Helped |
| replace Osprey+GPT3.5 with Ferret+Vicuna in the intention reasoner (ablation) | LucidDrag · DragBench | Mean Distance: 20.46 -> 22.49 | Hurt |
| replace Osprey+GPT3.5 with Ferret+LLama3 in the intention reasoner (ablation) | LucidDrag · DragBench | Mean Distance: 20.46 -> 21.96 | Hurt |
| replace Osprey+GPT3.5 with Ferret+GPT3.5 in the intention reasoner (ablation) | LucidDrag · DragBench | Mean Distance: 20.46 -> 20.65 | Hurt |
| replace Osprey+GPT3.5 with Osprey+Vicuna in the intention reasoner (ablation) | LucidDrag · DragBench | GScore: 7.37 -> 7.27 | Hurt |
Surprising findings
- FreeDrag and DragDiffusion achieve higher GScore on some individual categories, which the authors attribute to per-image model fine-tuning.
Limitations
- Complex objects are hard to drag, and long-distance drags can cause unexpected deformation.
- Results are sensitive to hyperparameters such as guidance weights.
Experimental setup
Datasets: DragBench LAION-style real images with captions (quality discriminator training)
Benchmarks: DragBench object moving test set (20 samples)
Metrics: Mean Distance GScore CLIP score LMM score
Baselines: DragDiffusion FreeDrag DragonDiffusion DiffEditor
Builds on: DragonDiffusion DragDiffusion DDIM classifier guidance
Closest papers by meaning
- FastDrag: Manipulate Anything in One Step (NeurIPS 2024 · similarity 0.89)
- CLIPDrag: Combining Text-based and Drag-based Instructions for Image Editing (ICLR 2025 · similarity 0.89)
- Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation (CVPR 2024 · similarity 0.88)
- FreeDrag: Feature Dragging for Reliable Point-based Image Editing (CVPR 2024 · similarity 0.88)
- GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models (ICLR 2025 · similarity 0.88)
- Text-Vision Co-Instructed Image Editing (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.