Research map / Image generation / Editing and style transfer
DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DragLoRA, which optimizes LoRA adapters online (with a DDS regularizer, a denoise-renoise input-feature adaptation, and an adaptive mode switch) for drag-based editing in Stable Diffusion 1.5.
Problem
Drag-based image editing with diffusion models is limited in accuracy by the low expressiveness of directly optimized DDIM-inverted latent features and in efficiency by large point-tracking search regions.
Approach
Replace direct latent optimization with online optimization of LoRA adapters, regularized by a reversed delta denoising score loss to stay close to the pretrained model, plus cyclic denoise-renoise input-feature adaptation and an adaptive switch that skips LoRA updates when point tracking is reliable.
Key results
- On DragBench, DragLoRA reaches MD 23.77 and m-MD 22.70 versus GoodDrag's 24.26 and 21.86, at 29.84 s versus GoodDrag's 56.97 s and DragDiffusion's 32.93 s.
- DragLoRA-Fast reduces time to 22.89 s per image while keeping MD 27.55 and 1-LPIPS 0.88.
- Ablation: adding DOO raises 1-LPIPS from 0.87 to 0.93 and cuts MD from 48.55 to 36.88; ILFA, EPT and ASS further reduce MD to 23.77.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add dual-objective optimization (DOO) (ablation) | DragLoRA baseline · DragBench | MD: 48.55 -> 36.88 | Helped |
| Add dual-objective optimization (DOO) (ablation) | DragLoRA baseline · DragBench | 1-LPIPS: 0.87 -> 0.93 | Helped |
| Add input latent feature adaptation (ILFA) (ablation) | DragLoRA with DOO · DragBench | MD: 36.88 -> 26.99 | Helped |
| Add efficient point tracking (EPT) (ablation) | DragLoRA with DOO and ILFA · DragBench | MD: 26.99 -> 25.45 | Helped |
| Add adaptive switching scheme (ASS) (ablation) | DragLoRA with DOO, ILFA, and EPT · DragBench | MD: 25.45 -> 23.77 | Helped |
| Use random-noise re-noising (SDS form) instead of DDIM re-noising (DDS form) in ILFA (ablation) | DragLoRA · DragBench | MD: 29.33 -> 23.77 | Helped |
| Use random-noise re-noising (SDS form) instead of DDIM re-noising (DDS form) in ILFA (ablation) | DragLoRA · DragBench | 1-LPIPS: 0.91 -> 0.87 | Hurt |
| Use angle-closer EPT instead of distance-closer EPT (ablation) | DragLoRA · DragBench | MD: 23.77 -> 27.55 | Hurt |
| Use angle-closer EPT instead of distance-closer EPT (ablation) | DragLoRA · DragBench | Time(s): 29.84 -> 22.89 | Helped |
| Use linear point search instead of distance-closer EPT (ablation) | DragLoRA · DragBench | MD: 23.77 -> 30.61 | Hurt |
| Use linear point search instead of distance-closer EPT (ablation) | DragLoRA · DragBench | Time(s): 29.84 -> 20.79 | Helped |
| Use DragLoRA instead of GoodDrag | DragBench drag editing · DragBench | MD: 24.26 -> 23.77 | Helped |
Surprising findings
- Handle points can be driven toward targets through input feature adaptation alone, without explicit LoRA gradient updates.
- Standard (ILFA-DDS style) renoising gave 1-LPIPS 0.91 but worse MD (29.33) than the SDS-form ILFA (0.87 / 23.77).
Limitations
- Quantitative metrics (MD, m-MD) are noted as indirect; no user study was conducted.
- Fails on challenging cases such as camera-motion edits that reveal unseen content (e.g., a face), producing low-fidelity outputs.
- m-MD can be optimistic under image distortion because the mask constrains handle points.
Experimental setup
Datasets: DragBench Drag100 VITON-HD
Benchmarks: DragBench Drag100 VITON-HD
Metrics: 1-LPIPS MD m-MD CLIP Time (s) minD
Baselines: DragDiffusion DragNoise StableDrag EasyDrag GoodDrag FreeDrag DragText AdaptiveDrag ClipDrag GDrag InstantDrag LightningDrag
Builds on: DragDiffusion DragGAN delta denoising score DDIM inversion LoRA
Closest papers by meaning
- DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing (CVPR 2024 · similarity 0.88)
- DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models (ICLR 2024 · similarity 0.87)
- GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models (ICLR 2025 · similarity 0.87)
- Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation (CVPR 2024 · similarity 0.86)
- FreeDrag: Feature Dragging for Reliable Point-based Image Editing (CVPR 2024 · similarity 0.86)
- LoRAtorio: An intrinsic approach to LoRA Skill Composition (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.