Research map / Image generation / Editing and style transfer
Move and Act: Enhanced Object Manipulation and Background Integrity for Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a tuning-free two-branch (inversion and editing) method that moves an edited object to a user-specified bounding box while preserving the background for consistent image editing with Stable Diffusion.
Problem
Consistent image editing of an object's action with existing three-branch methods (inversion, reconstruction, editing) lacks control over where the edited object is generated and often damages background details.
Approach
During inversion at step 35, latent updates using cross-attention losses move the object's attention into a user-given target box, inpaint the source area from its surroundings using UNet decoder features, and preserve the background; the editing stage then queries the inversion-stage self-attention keys and values, removing the reconstruction branch.
Key results
- Move&Act achieves CLIP-Score 0.3098 vs 0.3066 for MasaCtrl, 0.3035 for PnP and 0.2966 for InstructPix2Pix.
- Move&Act reports AP50 of 33.70 for generated position accuracy, a metric baselines cannot be scored on since they cannot control position.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Move&Act instead of MasaCtrl | Consistent image editing · 200 image-condition pairs | CLIP-Score: 0.3066 ± 0.0008 -> 0.3098 ± 0.0011 | Helped |
| Use Move&Act instead of PnP | Consistent image editing · 200 image-condition pairs | CLIP-Score: 0.3035 ± 0.0009 -> 0.3098 ± 0.0011 | Helped |
| Use Move&Act instead of InstructPix2Pix | Consistent image editing · 200 image-condition pairs | CLIP-Score: 0.2966 ± 0.0013 -> 0.3098 ± 0.0011 | Helped |
| Add source-area inpainting loss L_sai (ablation) | Move&Act · Object relocation and source-area inpainting | object information transfer | Helped |
| Add background preservation loss L_bg (ablation) | Move&Act · Background preservation | background consistency | Helped |
| Set the latent update step to 25 instead of 35 (ablation) | Move&Act · Object relocation and action editing | object information transfer | Hurt |
| Set the latent update step to 45 instead of 35 (ablation) | Move&Act · Object relocation and action editing | visual quality | Hurt |
| Use 25 latent-update iterations instead of 50 (ablation) | Move&Act · Background preservation and action consistency | visual quality | Hurt |
| Use 75 latent-update iterations instead of 50 (ablation) | Move&Act · Background preservation and action consistency | visual quality | Mixed |
| Use a larger starting step S for self-attention replacement (ablation) | Move&Act · Object appearance and background consistency | visual quality | Hurt |
| Replace keys and values only in UNet encoder self-attention layers (ablation) | Move&Act · Object appearance and background restoration | visual quality | Hurt |
Limitations
- Struggles with complex backgrounds surrounding the source area during inpainting.
- Performance is sensitive to the dilation kernel size used for the edge area.
Experimental setup
Datasets: 200 manually annotated condition-image pairs from the web
Benchmarks: 200 manually annotated condition-image pairs
Metrics: CLIP Score AP50 (YOLO Score)
Baselines: InstructPix2Pix PnP MasaCtrl
Builds on: Stable Diffusion MasaCtrl DIFT DDIM inversion
Closest papers by meaning
- Object-Aware Inversion and Reassembly for Image Editing (ICLR 2024 · similarity 0.89)
- Tuning-Free Inversion-Enhanced Control for Consistent Image Editing (AAAI 2024 · similarity 0.88)
- DesignEdit: Unify Spatial-Aware Image Editing via Training-free Inpainting with a Multi-Layered Latent Diffusion Framework (AAAI 2025 · similarity 0.85)
- PixelMan: Consistent Object Editing with Diffusion Models via Pixel Manipulation and Generation (AAAI 2025 · similarity 0.85)
- Balancing Preservation and Modification: A Region and Semantic Aware Metric for Instruction-Based Image Editing (ICML 2025 · similarity 0.85)
- PostEdit: Posterior Sampling for Efficient Zero-Shot Image Editing (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.