Research map / 3D vision / 3D generation
Image Sculpting: Precise Object Editing with 3D Geometry Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Image Sculpting lifts a single image object into 3D via NeRF/SDS reconstruction, deforms it with skeleton-based tools, and re-renders it into the original image with a DreamBooth, depth-ControlNet, and feature-injection SDXL enhancement pipeline.
Problem
Text-based and 2D drag-based image editing lack precise, quantifiable, 3D-aware control over object pose, rotation, translation, and composition, and 3D reconstructions from a single view are too coarse to be used directly as final images.
Approach
Combine single-view 3D reconstruction and classic geometry deformation (skeleton/linear blend skinning, ARAP) with a coarse-to-fine diffusion enhancement that uses one-shot DreamBooth for texture, depth ControlNet for geometry, and self-attention/feature injection from the coarse rendering, plus background-masked blend-in.
Key results
- On SculptingBench (28 images), the full method reaches DINO 0.853 vs 0.800 for SDEdit (t0=0.6) and 0.848 without feature injection.
- Full method achieves D-RMSE 1.99, lower than SDEdit (t0=0.4) at 1.71 only marginally higher, while keeping much better texture than SDEdit.
- Qualitative comparisons show 3DIT, DragDiffusion, ControlNet, InstructPix2Pix and DALL·E 3 fail on complex pose, rotation, and precise serial addition edits.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add feature injection (ablation) | Image Sculpting · SculptingBench | DINO: 0.848 -> 0.853 | Helped |
| add feature injection (ablation) | Image Sculpting · SculptingBench | D-RMSE: 2.33 -> 1.99 | Helped |
| add depth control (ablation) | Image Sculpting · SculptingBench | DINO: 0.851 -> 0.853 | Helped |
| add depth control (ablation) | Image Sculpting · SculptingBench | D-RMSE: 2.15 -> 1.99 | Helped |
Limitations
- Reconstruction from single view is coarse, so a separate enhancement stage is required
- Skeleton/deformation construction is manual and requires user interaction
- Evaluation is limited to a 28-image self-built benchmark with no human study
Experimental setup
Datasets: SculptingBench
Benchmarks: SculptingBench
Metrics: DINO score D-RMSE (depth-based geometric fidelity, MiDaS)
Baselines: OBJect-3DIT DragDiffusion ControlNet (openpose) InstructPix2Pix DALL·E 3 SDEdit DreamBooth
Builds on: Zero-1-to-3 Plug-and-Play DreamBooth SDXL ControlNet
Closest papers by meaning
- Thinking in Boxes: 3D Editing in Real Images Made Easy (NeurIPS 2026 · similarity 0.88)
- Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects (CVPR 2025 · similarity 0.87)
- PrEditor3D: Fast and Precise 3D Shape Editing (CVPR 2025 · similarity 0.87)
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.86)
- ID-Sculpt: ID-aware 3D Head Generation from Single In-the-wild Portrait Image (AAAI 2025 · similarity 0.86)
- 3DGS-Drag: Dragging Gaussians for Intuitive Point-Based 3D Editing (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.