Research map / 3D vision / 3D generation
StitchEdit: Stitching Depth Priors into Image Editors via Per-Layer Gradient Probing
Qin Guo, Dan Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StitchEdit, which transfers depth priors from a pretrained diffusion backbone into an image editor to improve spatially grounded text-guided edits, selecting layers via per-layer gradient probing.
Problem
Text-guided image editing with diffusion models struggles with spatial edits (object manipulation, viewpoint changes, occlusion handling) that require 3D scene structure understanding.
Approach
Uses per-layer gradient probing to identify which layers of a diffusion backbone carry geometric (depth) priors, and stitches those depth-relevant layers into an image editor to redirect Marigold-style priors toward editing.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune depth adapter with probe-guided per-layer schedule instead of uniformly (ablation) | StitchEdit · StitchEditBench | DCS: 0.80 -> 0.92 | Improved |
| add Stage 1 depth initialization (ablation) | editing LoRA · StitchEditBench | DCS: 0.70 -> 0.80 | Improved |
| replace hard top-K cutoff with probe-guided schedule (ablation) | StitchEdit · StitchEditBench | DCS: 0.87 -> 0.92 | Improved |
| use probe-derived layer ordering instead of random schedule (ablation) | StitchEdit · StitchEditBench | DCS: 0.83 -> 0.92 | Improved |
| use probe-computed boundaries instead of fixed cosine heuristic (ablation) | StitchEdit · StitchEditBench | DCS: 0.90 -> 0.92 | Improved |
| use cosine decay instead of step or linear (ablation) | StitchEdit · StitchEditBench | DCS: 0.88 / 0.90 -> 0.92 | Improved |
| train conflict-zone layers (l_s=30, l_e=55) (ablation) | StitchEdit · StitchEditBench | CLIP-Dir: 0.81 -> 0.75 | Hurt |
| narrow the compatible region (l_s=10) (ablation) | StitchEdit · StitchEditBench | DCS: 0.92 -> 0.86 | Hurt |
| condition on explicit source depth via ControlNet (ablation) | ControlNet-Depth · StitchEditBench | DCS / CLIP-Dir: 0.72 DCS, 0.62 CLIP-Dir | Hurt |
| use oracle target depth ControlNet instead of implicit prior (ablation) | ControlNet-Depth · StitchEditBench | DCS: 0.92 -> 0.90 | Hurt |
| use StitchEdit instead of instruction-based editors | StitchEdit · StitchEditBench | DCS: 0.74 -> 0.92 vs Nano-Banana | Improved |
| add Stage 1 depth LoRA to backbone | Qwen-Image-Edit · NYU Depth V2 | AbsRel: 12.2 -> 5.4 | Improved |
Experimental setup
Builds on: Marigold
Closest papers by meaning
- Diffusion Models are Geometry Critics: Single Image 3D Editing Using Pre-Trained Diffusion Priors (ECCV 2024 · similarity 0.91)
- Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceivers (CVPR 2026 · similarity 0.90)
- Zero-Shot Depth Aware Image Editing with Diffusion Models (ICCV 2025 · similarity 0.90)
- Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D (CVPR 2024 · similarity 0.89)
- ProEdit: Simple Progression is All You Need for High-Quality 3D Scene Editing (NeurIPS 2024 · similarity 0.88)
- SyncNoise: Geometrically Consistent Noise Prediction for Instruction-based 3D Editing (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
